AI-103 Questions 21-30: Generative AI Apps and RAG in Foundry

The biggest part of AI-103 is Implement generative AI and agentic solutions, worth 30-35% of the exam. Before you build agents, you need to be comfortable with the building blocks underneath them: connecting an app to a Foundry project, getting reliable output from models, tuning reasoning models, and grounding answers with retrieval-augmented generation (RAG).

This is the third post in our AI-103 series (Questions 21-30). Every question is an original practice scenario mapped to the official skills outline, not a copy of a real exam item. If you are new to the series, start with AI-103 Questions 1-10.

Pick your answer before you reveal it. Many wrong options on this exam are real features that solve a different problem, so read why each one does not fit.

Question 21: Connecting an App to a Foundry Project

A developer is writing a Python web app that must call models and agents in a Foundry project. The security team wants Microsoft Entra ID authentication and a single place to configure the connection. How should the app connect?

A) Copy each model deployment’s endpoint and API key into the app settings and call each one directly

B) Create an AIProjectClient with the project endpoint and DefaultAzureCredential, then call get_openai_client() to work with models and agents

C) Call the Azure Resource Manager endpoint at management.azure.com to send prompts to the project

D) Export the project as a template and run it inside the web app container

👁 Reveal Answer

Correct Answer: B

Explanation: A Foundry project has one endpoint in the form https://<resource-name>.services.ai.azure.com/api/projects/<project-name>. The Foundry SDK’s AIProjectClient uses that endpoint for project operations such as connections and tracing, and get_openai_client() returns an OpenAI-compatible client for the Responses API, agents, and evaluations. DefaultAzureCredential signs in with Microsoft Entra ID, using a managed identity in Azure or your Azure CLI login on a laptop. A works, but it scatters endpoints and keys across settings, which is exactly what the team wants to avoid. C is the control plane for managing resources, not for inference. D is not a real way to run a project.

Reference: Microsoft Foundry SDKs and endpoints

Question 22: JSON That Always Matches the Schema

An invoice app asks a model to return the vendor name, total, and due date as JSON. With JSON mode turned on, the output is always valid JSON, but sometimes a field is missing or an extra field appears, which breaks the parser. What should the developer do?

A) Keep JSON mode and add “always include every field” to the prompt

B) Raise the temperature so the model explores more output formats

C) Parse the output with a regular expression and drop unknown fields

D) Use structured outputs with a json_schema response format, strict set to true, every field listed as required, and additionalProperties set to false

👁 Reveal Answer

Correct Answer: D

Explanation: JSON mode guarantees valid JSON but not adherence to your schema. Structured outputs make the model follow the JSON Schema you supply. To use them, list every field as required (use a null type to make a field effectively optional) and set additionalProperties to false so no extra keys appear. A still relies on the model following a request. B makes output less predictable, not more. C hides the problem and cannot add a field that was never generated. Keep in mind that structured outputs do not support every scenario; for example, they cannot be combined with parallel function calls.

Reference: How to use structured outputs

Question 23: A Reasoning Model Rejects the Request

A team swaps a chat model for a reasoning model in existing code. Requests now fail with errors about unsupported parameters. The code sets temperature to 0.2 and max_tokens to 1,500. What should they change?

A) Remove temperature, replace max_tokens with max_completion_tokens (Chat Completions) or max_output_tokens (Responses API), and use reasoning_effort to control how much the model thinks

B) Set temperature to 0 instead of 0.2

C) Rename the system message to a developer message and keep the other parameters

D) Request a higher tokens-per-minute quota for the deployment

👁 Reveal Answer

Correct Answer: A

Explanation: Most reasoning models do not support sampling parameters such as temperature and top_p, and they do not support max_tokens. Use max_completion_tokens or max_output_tokens instead; both limits include the hidden reasoning tokens, so set them with room to spare. reasoning_effort trades answer depth against speed and cost, and the values each model accepts vary. B still sends an unsupported parameter. C is harmless, because developer messages work the same as system messages, but it does not fix the error. D addresses throttling, not invalid parameters.

Reference: Azure OpenAI reasoning models

Question 24: Vector Search Misses Exact Part Numbers

A support assistant uses pure vector search over product manuals. Questions such as “how do I reset the pump?” work well, but questions that contain exact part numbers like “XR-4471-B” often return manuals for similar-sounding products. What is the best fix?

A) Increase k on the vector query from 10 to 100

B) Switch to a larger embedding model

C) Send a hybrid query that runs keyword and vector search together and merges the results with Reciprocal Rank Fusion (RRF)

D) Remove part numbers from the indexed content so they cannot confuse the embeddings

👁 Reveal Answer

Correct Answer: C

Explanation: Vector search finds conceptually similar text, but exact strings such as product codes, jargon, dates, and names are matched better by keyword (BM25) search. A hybrid query includes both search and vectorQueries in one request, runs them in parallel, and merges the two result lists with RRF, so an exact code match still ranks high. A only adds more loosely related results. B may improve similarity but still does not guarantee exact string matches. D removes the very information users search for.

Reference: Hybrid search using vectors and full text in Azure AI Search

Question 25: The Right Chunk Is Found, but Ranked Too Low

Testing shows that the chunk with the correct answer is almost always somewhere in the top 50 results of a hybrid query, but often not in the top 5 that the app sends to the model. What should the team add?

A) A scoring profile that boosts newer documents

B) Semantic ranker, with a semantic configuration that lists the title and content fields in priority order, and queryType set to semantic

C) Send all 50 results to the model on every request

D) Rebuild the index with chunks twice as large

👁 Reveal Answer

Correct Answer: B

Explanation: Semantic ranker adds a second ranking pass over an existing result set scored by BM25 or RRF. It takes up to the top 50 results and reorders them with deep learning models adapted from Bing, which is exactly the “found but ranked too low” problem. Its input is built from the title, keywords, and content fields in your semantic configuration, so field order matters. A helps only if freshness is the issue. C raises cost and latency and gives the model more irrelevant text to get confused by. D changes what is retrieved but does not fix the ordering.

Reference: Semantic ranking in Azure AI Search

Question 26: Long Manuals Lose Their Second Half

An indexer generates one embedding per document with text-embedding-3-small. Some equipment manuals are 40,000 tokens long, and questions about their later chapters never find a match. What should the team do?

A) Split each document into chunks, for example about 512 tokens with some overlap, by using the Text Split skill, and embed each chunk

B) Compress the PDF files before indexing

C) Switch the chat model to one with a larger context window

D) Index only the first page of each manual, which usually holds a summary

👁 Reveal Answer

Correct Answer: A

Explanation: Embedding models have an input limit; for text-embedding-3-small it is 8,191 tokens, so text beyond that is lost when a whole manual is embedded as one vector. Chunking keeps each piece under the limit and also gives sharper vectors for focused topics. Microsoft suggests starting around 512 tokens with overlap so that sentences at chunk edges keep their context, and the Text Split skill does this inside the indexing pipeline. B changes file size, not token count. C affects the chat model, not the embeddings used for retrieval. D throws away most of the content.

Reference: Chunk large documents for vector search solutions

Question 27: Classifying a Million Tickets a Day

A company needs to sort about a million short support tickets a day into 12 categories. Answers must come back in under a second, and cost per ticket matters a lot. Tests show the task is simple for most models. Which approach should the team evaluate first?

A) The largest reasoning model available, with reasoning effort set to high

B) An image generation model, so screenshots in tickets are handled too

C) A text embedding model, asked to write the category name as its output

D) A small language model, measured against a labeled sample of tickets before going live

👁 Reveal Answer

Correct Answer: D

Explanation: AI-103 expects you to match the model to the task. A simple, high-volume classification job with strict latency and cost targets is a natural fit for a small language model, as long as you confirm its accuracy on labeled data first. A would work but spends far more time and money per ticket than the task needs. B generates images; it does not classify text. C is a trap: embedding models return vectors, not text, so they cannot “write” a category (although you could build a nearest-neighbor classifier on top of embeddings, that is not what this option describes).

Reference: AI-103 study guide: choose an appropriate model for each task

Question 28: Letting the Model Check Its Own Work

A drafting assistant writes policy summaries that sometimes skip required sections or state numbers that do not appear in the source policy. The team wants the app to catch and fix these problems before a person sees the draft. Which design fits?

A) Raise the temperature so the model writes more varied drafts

B) Add “be accurate and complete” to the end of the prompt

C) Add a self-critique loop: a second model call checks the draft against the source and a checklist of required sections, and the draft is revised from that feedback, with a fixed maximum number of rounds

D) Show the model’s full internal reasoning to the end user so they can spot mistakes

👁 Reveal Answer

Correct Answer: C

Explanation: The outline asks you to implement model reflection and self-critique loops. A separate critique step with a clear rubric (required sections present, every number traceable to the source) finds specific problems, and a revision step fixes them. Capping the rounds keeps cost and latency predictable, and you can still send drafts that fail the check to a person. A adds randomness, not accuracy. B is a vague instruction the model was already trying to follow. D pushes quality control onto users, and reasoning models only offer optional reasoning summaries, not their raw reasoning.

Reference: Azure OpenAI reasoning models (reasoning summaries)

Question 29: Summaries That Never Look the Same Twice

A non-reasoning chat model writes meeting summaries. Users complain that the structure changes every time: sometimes bullets, sometimes paragraphs, and action items appear in different places. What is the quickest, cheapest fix to try first?

A) Put an explicit output template in the system message and add two or three short example summaries (few-shot examples) that follow it

B) Fine-tune a model on 10,000 past summaries

C) Increase top_p to 1.0

D) Increase max_tokens so the model has room for every section

👁 Reveal Answer

Correct Answer: A

Explanation: Prompt engineering is the first tool for tuning generation behavior. A clear template (for example: Summary, Decisions, Action items with owner and date) plus a few examples shows the model exactly what “good” looks like, and lowering the temperature on a non-reasoning model can make the result even more consistent. B may work, but it costs time and money and should come after prompt changes fail. C widens the range of tokens the model samples from, which adds variety. D only matters if summaries are being cut off.

Reference: AI-103 study guide: tune generation behavior

Question 30: Blocking a Worse Agent Version Before Release

A team updates its Foundry agent’s instructions and tools every sprint by using Azure Pipelines. They want each new version scored against the current version on a test dataset, with a clear signal about whether differences are real or just random variation, before release. What should they add?

A) A monthly manual review in the Foundry playground

B) The AI Agent Evaluation task in the pipeline, with the new and current agent versions, a dataset of test queries, and the evaluators to run

C) Stronger content filters on the production deployment

D) Continuous evaluation of production traffic only, after release

👁 Reveal Answer

Correct Answer: B

Explanation: The AI Agent Evaluation extension for Azure DevOps (currently in preview) adds the AIAgentEvaluation task to a pipeline. You pass the project endpoint, a judge model deployment, a dataset file that lists the evaluators and test queries, and agent IDs such as my-agent:1,my-agent:2, with an optional baseline. The pipeline summary shows scores with confidence intervals and a statistical comparison, so you can tell a real regression from noise. A is too slow and too infrequent for sprint releases. C reduces harmful content but measures nothing about quality. D is useful, but it only finds problems after users have seen them. A GitHub Actions version of this evaluation also exists.

Reference: Run agent evaluations in Azure DevOps

Study Tips for Generative AI Apps and RAG

• Know the two clients. AIProjectClient handles the project itself, and get_openai_client() handles models, agents, and evaluations.
• Reasoning models have their own parameters. No temperature or max_tokens; use max_completion_tokens or max_output_tokens and reasoning_effort.
• Retrieval is a stack. Chunk the content, then use hybrid search to find candidates, then semantic ranker to reorder them, then send the best few to the model.
• Fix prompts before you fine-tune. Templates, examples, and structured outputs solve most format problems quickly.
• Measure before you ship. Put evaluations in the pipeline so a worse version never reaches users.

Keep Practicing for AI-103

Review planning, security, and monitoring in AI-103 Questions 11-20, then continue to Questions 31-40, which go deeper into agents: tools, memory, and multi-agent orchestration. The official AI-103 study guide lists every skill measured.