recall&build

BUILD IT. EXPLAIN IT. DEFEND THE DECISION.

Bring evidence
to the interview.

Ten practical challenges for your support-agent project. Start with a two-minute answer. Test your reasoning in a lab. Come back with what actually happened.

Start a drill Download your worksheet

Original practice scenarios inspired by seven linked reading guides. No promises about what a particular employer will ask.

01 / EXPLAIN

Answer before opening the checklist.

02 / BUILD

Run the exercise in your laptop terminal.

03 / VERIFY

Keep a trace, test result or measurement.

04 / RECALL

Revisit the concepts in your study deck.

YOUR PRACTICE SEQUENCE

Decisions worth rehearsing.

Agent lab destination

Shared concepts across both paths. Provider selection changes the agent lab links. Drill answers and checklist use are not saved here; signed-in lab checkpoints use the existing progress system.

01

Retrieval / 3 · Add knowledge

Find where the evidence disappeared

A support answer contradicts a policy you know is in the document collection. What would you inspect before changing the model?

Compare your answer with the checkpoints
  • Check the indexed revision and authorized document scope.
  • Distinguish a missing candidate from a candidate removed during ranking or context assembly.
  • If the passage reaches the model, examine how the answer uses it.

Follow-up: What if the expected document belongs to another customer?

Put it into practice

Choose one failing query. Trace its expected passage through ingestion, access filters, candidates, ranking and the final context. Change only the failing stage.

Bring back: A before/after trace with document IDs, the diagnosed stage and a regression case.

Open the related lab Extension exercise; not a separately tracked lab.

Check the technical guidance

Microsoft: RAG architecture

Further reading: PracHub · InfoWok

02

Retrieval / 3 · Add knowledge

Keep the exception with the rule

A refund policy puts its exceptions in a table below the main rule. Your chunks separate them. How will you choose a better split?

Compare your answer with the checkpoints
  • Preserve useful document structure and source identifiers.
  • Treat size and overlap as experiment settings, not universal constants.
  • Keep evaluation questions fixed while comparing alternatives.

Follow-up: Could increasing overlap improve recall while making the answer more expensive?

Put it into practice

Compare two chunk sizes and one structure-aware split on the same small labeled question set. Include a question that needs the table.

Bring back: A comparison of retrieval recall, answer support and context tokens, plus one failed example.

Open the related lab Extension exercise; not a separately tracked lab.

Check the technical guidance

Microsoft: document chunking

Further reading: techinterview.org

03

Retrieval / 3 · Add knowledge

Find both product codes and paraphrases

Exact product-code queries work well, but descriptions in everyday language miss the right page. Should you add vectors, fusion or reranking?

Compare your answer with the checkpoints
  • Explain the different signals supplied by text and vector search.
  • Keep fusion and reranking distinct.
  • A reranker cannot rescue a relevant document absent from its candidates.

Follow-up: When would the extra ranking step be too costly?

Put it into practice

Use the retrieval lab to compare lexical, dense and fused rankings. Add a reranker only as a measured extension.

Bring back: Recall and ranking results split by exact identifiers and paraphrases, with retrieval latency.

Open the related lab Extension exercise; not a separately tracked lab.

Check the technical guidance

Microsoft: hybrid search

Further reading: AIJobPrep

04

Agents / 5 · Add tools

Justify the autonomy

A ticket process has fixed validation and approval steps, but some requests need unpredictable information gathering. Which parts should an agent control?

Compare your answer with the checkpoints
  • Keep known business rules in application code.
  • Name the uncertainty that justifies dynamic tool selection.
  • Define explicit stopping and escalation conditions.

Follow-up: What would make you replace the agent with a workflow?

Put it into practice

Draw a fixed workflow and an agent variant for the same support task. Run representative cases using your selected provider’s existing agent lab.

Bring back: A decision note comparing success, number of calls, permissions and failure handling.

Open OpenAI agent lab Extension exercise; not a separately tracked lab.

Check the technical guidance

Anthropic: building effective agents

Further reading: PracHub · InfoWok

05

Agents / 6 · Handle failures

Stop a loop without repeating an action

A ticket tool times out after writing successfully. The agent repeatedly calls it because it never saw confirmation. How should the application recover?

Compare your answer with the checkpoints
  • Separate a failed operation from an unknown result.
  • Bound elapsed time, turns and spending independently.
  • Check the system of record before repeating a consequential action.

Follow-up: What should the learner see if reconciliation is also unavailable?

Put it into practice

In a local fixture, lose a successful tool response. Add bounded attempts and reconcile the stored result using a stable operation ID before retrying.

Bring back: A failure trace, a stop reason and an assertion that only one ticket was created.

Open OpenAI agent lab Extension exercise; not a separately tracked lab.

Check the technical guidance

Anthropic: building effective agentsOpenAI: agent safety

Further reading: techinterview.org · InfoWok

06

Evaluation / 4 · Evaluate and observe

Decide whether a change can ship

A new prompt improves average answers but fails more often when no supporting document exists. Would you release it?

Compare your answer with the checkpoints
  • Separate development examples from the held-out test set.
  • Combine executable checks with calibrated human or model judgments.
  • Inspect repeated trials and important failure slices rather than relying on one average.

Follow-up: How would you detect that your judge rewards longer answers?

Put it into practice

Add answerable, unsupported and adversarial slices to the evaluation lab. Compare the baseline and candidate with predetermined release criteria.

Bring back: A slice-level report including failures, latency, cost and a written release decision.

Open the related lab Extension exercise; not a separately tracked lab.

Check the technical guidance

OpenAI: evaluation best practices

Further reading: PracHub · AIJobPrep

07

Evaluation / 7 · Test the system

Check the action behind the answer

The agent says it resolved a ticket and its final message looks correct. Its trace shows a forbidden tool attempt. Is the run successful?

Compare your answer with the checkpoints
  • Define permitted steps as well as the required final state.
  • Score tool choice, arguments and approval compliance.
  • Verify completion in the target system; a fluent claim is insufficient.

Follow-up: Can a blocked unsafe attempt still count as a quality failure?

Put it into practice

Replay allowed and disallowed tool paths in the agent lab. Check tool arguments, approvals and stored outcomes separately from the final text.

Bring back: A trace checklist and tests for an allowed action, a blocked action and an unsupported success claim.

Open OpenAI agent lab Extension exercise; not a separately tracked lab.

Check the technical guidance

OpenAI: trace gradingOpenAI: agent safety

Further reading: PracHub

08

Reliability / 1 and 6 · API contract and reliability

Validate meaning after validating shape

A model returns a perfectly valid ticket object containing a nonexistent account and an unsupported priority. What can your API safely accept?

Compare your answer with the checkpoints
  • Use schema validation for shape and separate checks for business constraints.
  • Keep authorization outside the model’s decision.
  • Handle refusals and incomplete output explicitly.

Follow-up: Which fields should trusted server context supply instead of the model?

Put it into practice

Extend the ticket lab with schema-valid but business-invalid fixtures. Exercise refusal, missing output and timeout paths.

Bring back: Boundary tests showing which inputs are rejected and that no unauthorized action runs.

Open the related lab Extension exercise; not a separately tracked lab.

Check the technical guidance

OpenAI: structured outputsOpenAI: agent safety

Further reading: PracHub

09

Performance / 9 and 10 · Load and optimize

Optimize the slow stage

Your support agent meets its quality target, but tail latency and cost per resolved ticket are too high. Which change do you test first?

Compare your answer with the checkpoints
  • Find the bottleneck before choosing an optimization.
  • Parallelize only independent work and distinguish streaming responsiveness from completion time.
  • Check quality after reducing context, calls or model size.

Follow-up: Which requests can tolerate deferred batch processing, and which cannot?

Put it into practice

Measure retrieval, queue, model and tool time. Compare one routing or call-reduction change on the same evaluation set. Use the cache lab for a separate cache experiment.

Bring back: A baseline/candidate report for task success, p95 latency and cost per successful task, including retries.

Open the related lab Extension exercise; not a separately tracked lab.

Check the technical guidance

OpenAI: latency optimizationOpenAI: evaluation best practices

Further reading: ProductionAIEngineer · Aced, formerly Exponent

10

Production / Portfolio · Explain your decisions

Explain a failure you can prove

A stakeholder asks what you personally built, what failed and how you know the fix helped. Prepare an answer backed by your project artifacts.

Compare your answer with the checkpoints
  • Separate your contribution from team work.
  • State whether the evidence is local, simulated, deployed or from real users.
  • Name one unresolved limitation and the next useful experiment.

Follow-up: What would you do if a launch decision had to be made with incomplete evidence?

Put it into practice

Choose one real lab failure. Record the symptom, baseline, investigation, smallest fix and rerun. Label local simulations honestly.

Bring back: A two-minute explanation linking your commit, failing case and before/after measurements.

Open the related lab Extension exercise; not a separately tracked lab.

Check the technical guidance

OpenAI: evaluation best practices

Further reading: Cubitrek · ProductionAIEngineer

KEEP THE CONCEPTS FRESH

Practice the ideas behind your answer.

Use the existing recall deck between builds. A fluent explanation and a passing test are different kinds of evidence; keep both.

Review OpenAI agent cards Review all concepts

SOURCE NOTES / CHECKED 5 OCTOBER 2026

Read beyond the checklist.

We followed all seven resource links in Shirin Khosravi Jam’s post. These are independent preparation guides, not official employer question banks. Technical checkpoints link to primary documentation above.

InfoWok

A round-by-round reading guide. Use its question themes; the interview-frequency statistics are not independently established.

PracHub

Useful failure-analysis and answer-structure prompts. Its SCORE framework is the publisher’s practice rubric, not an employer standard.

techinterview.org

Practical debugging and customer-discovery scenarios. Treat descriptions of hiring rounds as the publisher’s account.

AIJobPrep

Additional prompts on retrieval, memory and cost. Technical answers still need checking against provider documentation.

Aced, formerly Exponent

Broader design prompts including batching, uploads and voice. Company attributions are candidate reports according to the publisher, not employer-confirmed questions. Some linked material may require membership.

ProductionAIEngineer

A supplementary guide to discussing operational tradeoffs and presenting a project. Example numbers are not performance targets.

Cubitrek

A wider checklist for reflecting on experience. We do not adopt its score-to-seniority thresholds or require learners to invent production experience.

Batching systems, multimodal uploads and real-time voice are useful advanced design prompts from the reading. Dedicated runnable labs for those topics are not included yet. Reading a guide does not automatically mark progress complete.