Your AI app answered.
See what happened inside.

A working application produces a structured incident note and uses an external service when needed.

• the incident-note assistant
• prompts, messages, models, structured output, and tools
• dependencies, credentials, and the service-status API
• turn an incident description into a structured summary
• let a small agent look up service status through a tool
• handle an unavailable service and verify the response

Different inputs follow different paths through a small, understandable workflow.

• a workflow with two routes
• state, nodes, edges, conditional routing, and bounded execution
• the graph, tools, and sample incident inputs
• classify an incident and choose the next step
• check service status and handle a tool failure
• trigger both routes and inspect the final state

Learners identify which step caused a request to fail or become slow.

• follow one request in LangSmith
• trace trees, child runs, metadata, tags, and errors
• tracing credentials, project configuration, and a first trace
• inspect model inputs, outputs, tool calls, and latency
• introduce a slow tool and compare tagged prompt variants
• locate a failed request and record a quality annotation

Learners can inspect a conversation and relate a prompt change to observed usage and quality feedback.

• the observable chatbot
• observations, sessions, token usage, estimated cost, prompts, and scores
• connect the SDK and verify the first recorded interaction
• inspect a conversation, model calls, latency, and usage
• fetch a versioned prompt and attach simple user feedback
• compare two prompt versions on the same sample inputs

Learners distinguish a retrieval problem from a generation problem using trace evidence.

• question, retrieval, and answer
• RAG steps, retrieval failures, OpenInference, and annotations
• launch Phoenix, load sample runbooks, and enable instrumentation
• inspect retrieval and generation spans
• supply irrelevant context and annotate the resulting answer
• improve retrieval and compare the before-and-after traces

Two applications appear in one connected trace with the slow or failed operation identifiable.

• one request across two applications
• SDKs, spans, attributes, context propagation, OTLP, and the Collector
• start both services, the Collector, and the trace backend
• create custom spans and preserve trace context over HTTP
• add a tool delay, record an exception, and find the failing span
• add one useful attribute and verify the complete trace

Learners produce an evaluation table and justify whether a change improved the application.

• a small evaluation report
• faithfulness, response relevancy, context precision, and reference data
• configure the evaluator and inspect the dataset fields
• score the baseline responses and interpret the results
• change context or prompts and compare against the baseline
• review a misleading score and apply a simple quality threshold

Learners query service health and explain what changed during a controlled failure.

• watch application metrics change
• counters, gauges, histograms, labels, scraping, and cardinality
• start the application, expose metrics, and configure Prometheus
• measure requests, failures, latency, in-flight requests, and token totals
• request rate, error ratio, and approximate p95 latency with PromQL
• generate slow and failed requests and verify the changes

A working dashboard reacts to traffic and a basic alert transitions through firing and resolved states.

• the finished operations dashboard
• data sources, queries, panels, variables, thresholds, and alerts
• start the bundled services and connect the Prometheus data source
• build panels for traffic, errors, latency, and token usage
• add a service filter and an error-rate alert; trigger its firing state
• restore the service, verify resolution, and export the dashboard

Learners move from an error log to the corresponding request trace and explain the failure.

• locate a failed request in logs
• structured logs, severity, labels, LogQL, and trace correlation
• start the application, Alloy, Loki, Grafana, and the supplied trace backend
• generate successful and failed requests and query the logs
• find a request ID in logs and use its trace ID to inspect the trace
• diagnose a timeout and redact one sensitive field