How to assess · For hiring teams

How to Assess DevOps Skills When Hiring

The test formats that actually work for DevOps, what a strong answer looks like, sample questions and a scoring rubric you can use as-is.

The short answer

Assess DevOps with a task, not a conversation: incident scenario walkthrough, pipeline review and improvement, ai-scored assessment (e.g. cohesyve) or small practical: containerise and deploy. Score it against written criteria you fix before you see any submissions, and weight the criteria that the role actually depends on.

Makes deployments repeatable and reversible before making them fast
Treats infrastructure as code with review, not as console clicks with a wiki page
Designs alerts around symptoms users would notice, not every metric that can be graphed
Approaches an incident by establishing what changed and what the blast radius is, before theorising

Build a DevOps assessment →Read the full article ↓

Paste a job description; Cohesyve generates a role-specific assessment and rubric. Ten candidates free, no card.

DevOps is one of the hardest skills to assess because the title covers everything from writing YAML to redesigning a delivery organisation. What you are actually hiring for, in almost every case, is judgement: what to automate first, how to make deployments boring, how to find the cause of an outage under pressure, and how to change a system without breaking the people who rely on it. None of that is visible on a certifications list. This page covers how to assess DevOps skill as it is practised: pipelines, infrastructure, observability, incident response and the ability to reason about a system you did not build.

Why DevOps is worth testing

A weak DevOps hire does not fail visibly. Deployments still happen; they just take longer, break more often, and get worked around. The pipeline grows scripts nobody understands, the alerts fire constantly and get ignored, and the one person who knows how production works becomes a bottleneck. Testing surfaces whether a candidate reduces that kind of entropy or adds to it, and that determines how much engineering time the whole team keeps.

What strong DevOps looks like

Makes deployments repeatable and reversible before making them fast
Treats infrastructure as code with review, not as console clicks with a wiki page
Designs alerts around symptoms users would notice, not every metric that can be graphed
Approaches an incident by establishing what changed and what the blast radius is, before theorising
Knows the difference between a rollback, a roll-forward and a feature flag, and when each applies
Thinks about secrets, least privilege and the audit trail as part of the pipeline, not as a separate project
Writes runbooks and automates them away over time

Ways to assess DevOps

Incident scenario walkthrough

Present a realistic incident: a deployment went out, error rate is up 8%, and the on-call engineer has a dashboard, logs and the deploy history. Ask the candidate to talk through what they do in the first fifteen minutes and what they would change afterwards.

Pros

Tests diagnostic method and judgement under the conditions that matter most; hard to fake.

Cons

Talk-based; pair with a hands-on task for mid-level roles.

Best for Any level; the depth of the follow-up questions scales it.

Pipeline review and improvement

Provide a real-looking CI/CD configuration with problems — secrets in plain text, no rollback path, tests that run after deploy, a 40-minute build with no caching. Ask what they would fix, in what order, and why.

Pros

Concrete, fast and directly relevant; prioritisation reveals experience.

Cons

Tool-specific syntax can distract; tell them to ignore syntax and focus on design.

Best for Mid and senior roles owning delivery pipelines.

AI-scored assessment (e.g. Cohesyve)

Generate a DevOps scenario from your job description — an incident, a pipeline review, a migration plan — with a rubric. Each candidate receives a different variant; reasoning is scored in writing.

Pros

Asynchronous and consistent; DevOps judgement is largely written reasoning, which scores well; a different scenario per candidate.

Cons

Cannot observe hands-on tool use; have finalists do a short practical.

Best for Screening a large pool for judgement before technical interviews.

Small practical: containerise and deploy

Give a simple application and ask for a Dockerfile, a minimal pipeline definition and a short note on how they would run it in production. Ninety minutes capped.

Pros

Confirms hands-on fluency and shows what they consider "production-ready".

Cons

Favours candidates who have done exactly this recently; less useful for platform-design roles.

Best for Mid-level engineers who will implement rather than design.

Cohesyve

Run a DevOps assessment on your next opening

Cohesyve generates a unique DevOps task per candidate from your job description, with the scoring rubric attached. Questions are different for every applicant, so they cannot be shared or looked up.

Try it free →Book a 30-min demo

What to test

Delivery pipelines

Whether they can make deployment safe and fast, in that order.

Order the fixes to a broken pipeline by risk reducedDesign a rollback that takes under two minutesExplain where tests, security scans and approvals belong in the flow

Infrastructure as code

Whether infrastructure is reviewable and reproducible.

Review a Terraform or CloudFormation change and spot the destructive actionExplain how state is managed and what goes wrong when it is notDescribe how they would bring a hand-built environment under code

Observability and alerting

Whether they know what to measure and what to wake someone for.

Choose four alerts for a web service and justify eachExplain the difference between a log, a metric and a trace, and when each is the right toolReduce an alert set that fires forty times a day to one that is trustworthy

Incident response

Whether they can find cause and limit damage under pressure.

Walk through the first fifteen minutes of a deployment-caused outageDecide between rollback and roll-forward for a given failureWrite a blameless post-mortem outline with actions that will actually happen

Sample DevOps questions

A deployment just went out and error rate doubled. What do you do first?

Entry

Look for Confirm the correlation with the deploy, assess blast radius, roll back if the path is safe, then investigate — not investigate first.

Here is a pipeline that deploys and then runs tests. What is wrong and how would you restructure it?

Entry

Look for Tests before deploy, gating, and a sense of what should block versus warn.

How would you store and rotate secrets for a pipeline that deploys to three environments?

Mid

Look for A secrets manager, no secrets in the repo or logs, per-environment scoping, rotation without downtime.

Your alerts fire constantly and the team ignores them. Fix it.

Mid

Look for Alert on user-facing symptoms and SLOs, delete or downgrade the rest, add ownership and runbooks.

Plan a migration of a stateful service to a new cluster with under five minutes of downtime.

Senior

Look for Data replication strategy, cutover plan, rollback point, rehearsal, and who is on the call.

Red flags

Describes production changes made by hand in a console with no record
Has no answer for how a bad deployment is reversed
Alerts on everything and cannot say which alerts matter
Blames people rather than systems when describing past incidents
Treats security and secrets as someone else's concern

Scoring rubric

Criterion	Weight	What strong looks like
Incident judgement	30%	Contains damage first, diagnoses methodically, learns without blame.
Delivery design	25%	Pipelines are safe, reversible and understood; fast comes after.
Infrastructure discipline	20%	Everything is code, reviewed, and reproducible.
Observability	15%	Measures what users experience; alerts are few and trusted.
Security awareness	10%	Least privilege and secrets handling are built in.

Mistakes hiring teams make

Testing tool trivia (flags, syntax) instead of judgement about systems
Hiring on certifications without a scenario
Skipping the incident question because the candidate is not applying for on-call
Not asking how they would reverse a change
Confusing "has used Kubernetes" with "can run a service safely"

Roles that need DevOps

DevOps EngineerSite Reliability EngineerPlatform EngineerCloud EngineerInfrastructure EngineerRelease Engineer

Common questions

How do I assess DevOps without a live environment?

Use scenarios and artefacts: an incident narrative with a dashboard screenshot, a pipeline definition to review, a Terraform diff. Judgement shows in how candidates reason about these; hands-on fluency can be confirmed in a short practical for finalists.

Should I require specific tools like Kubernetes or Terraform?

Require what the role uses daily, test judgement generally. Tool syntax is learnable in weeks; knowing what to automate, alert on and reverse is not.

What separates a senior DevOps engineer from a mid-level one?

Seniors design systems that stay safe as the team changes — pipelines others can understand, alerts others trust, incident processes that improve. Mid-level engineers execute those designs well.

Is a DevOps assessment relevant for backend developers?

A lighter version, yes. Any developer who ships to production should be able to describe how their code is deployed, monitored and rolled back. A single incident question covers it.

Cohesyve · Skill assessments for hiring

Test DevOps before the first interview

Generate a role-specific DevOps assessment from your job description and see who can do the work before you spend interview time on them.

1,500+

assessments completed

50%

faster time-to-hire

90%

completion rate

5 min

from JD to assessment

No credit card · 10 free candidates · Plans sized to your hiring volume

From the blog

Related skill guides