Generative AI Developer Interview Questions (With What to Listen For)

Good generative AI developer interview questions probe five areas: model selection, grounding with retrieval, fine-tuning decisions, evaluation and safety. Strong candidates answer with trade-offs and examples, not buzzwords.

Model selection

Q: How do you choose between a large commercial model and a smaller open-source model? Listen for: cost, latency, privacy, accuracy on the task, ability to self-host, and testing several models against the same evaluation set.

Grounding and retrieval

Q: Users say the assistant makes things up. What do you do? Listen for: retrieval quality checks, citations, constraining answers to sources, refusing when unsure, and measuring hallucination rates.

Q: How do you chunk long documents? Listen for: structure-aware chunking, overlap, metadata, and testing different strategies.

Fine-tuning

Q: When is fine-tuning worth it? Listen for: consistent style or format, domain vocabulary, lower latency or cost with smaller models — and that prompting plus retrieval should be tried first.

Evaluation

Q: How would you evaluate a customer-support assistant? Listen for: a golden set of real questions, rubric-based scoring, human review for edge cases, and regression testing before every change.

Safety and privacy

Q: How do you defend against prompt injection? Listen for: separating instructions from content, limiting tool permissions, output validation and monitoring.

Q: How do you handle personal data? Listen for: redaction, retention settings, access controls and compliance awareness.

Cost and performance

Q: Our monthly LLM bill doubled. Where do you look? Listen for: caching, shorter prompts, model routing, batching and usage analytics.

Practical exercise

Ask for a tiny content-generation tool that follows a style guide, with five test inputs and a scoring rubric. It shows prompting, structure and evaluation habits.

Explore our vetted generative AI developers.