How to hire an automation agency: the complete vetting playbook

Why hiring an automation agency is hard right now

The automation agency market in 2026 has three structural problems that make hiring harder than it should be.

First, the credential gap. There's no equivalent of a CPA or a bar exam for automation work. Anyone can call themselves an "automation expert" or "AI consultant" and start taking client money. The Make.com partner directory has thousands of "certified" partners, but the certification means they passed an online quiz, not that they've successfully delivered enterprise automation projects. Same for Zapier Experts. The credentialing systems are marketing programs, not quality signals.

Second, the AI gold rush. Since late 2023, the rise of generative AI has created a flood of new agencies positioning themselves as "AI automation experts." Many of these agencies are 6-18 months old, run by people with no traditional automation or systems integration background. They're good at marketing AI capabilities. They're often bad at executing reliable production automations. Distinguishing legitimate AI-forward agencies from AI-marketing agencies takes work.

Third, the freelancer arbitrage. A growing number of "agencies" are actually solo operators or 2-3 person teams reselling work from offshore freelancers. They charge agency rates ($125-$200/hr) and pay subcontractors $20-$40/hr, pocketing the spread. The work product reflects the actual builders, not the marketing material. The agency you see in the sales process is not the agency that builds your automation.

These three problems compound. The credential gap means you can't filter on certifications. The AI rush means you can't filter on stated expertise. The freelancer arbitrage means you can't filter on the sales-stage presentation. You have to vet based on demonstrated work and verifiable references.

The good news: there are real automation agencies in 2026 doing excellent work. They tend to have 5+ years of operating history, named technical leadership, named team members on their site, specific industry or technology focus, and verifiable client references in your stack. The framework in this guide helps you find them and filter out the rest.

The four kinds of "automation agencies" in 2026

Not all agencies positioning as automation builders are doing the same work. Understanding the four categories helps you target your search.

Category 1: Workflow platform resellers. These agencies build on top of Zapier, Make, n8n, or similar platforms. They're skilled at platform-specific workflow design. They typically don't write custom code. They're a good fit for small-to-mid business automation where the platform's native capabilities cover the use case. They're a bad fit for anything requiring custom logic, high-volume processing, or integration with systems that don't have pre-built platform connectors. Hourly rates typically $75-$150. Project sizes typically $5K-$30K.

Category 2: SaaS-specific integration shops. These agencies specialize in a specific SaaS ecosystem — Salesforce, HubSpot, NetSuite, ServiceNow. They have deep expertise in one platform's API, customization model, and integration patterns. They're excellent for projects centered on that platform. They're less useful for multi-system orchestration outside their specialty. Hourly rates typically $150-$250. Project sizes typically $25K-$200K.

Category 3: Custom development agencies with automation practice. These are traditional software development agencies that have automation work as one of their service lines. They write code, build custom integrations, and handle infrastructure. They're a good fit for complex multi-system projects, high-volume use cases, or anything requiring real engineering. They're overkill (and expensive) for simple workflow automation. Hourly rates typically $175-$300. Project sizes typically $50K-$500K+.

Category 4: AI-specialist consultancies. Newer category that grew rapidly in 2024-2025. These agencies focus on LLM-based automation, AI agents, document AI, voice AI. Quality varies wildly — some have legitimate AI/ML expertise, others are workflow platform resellers with OpenAI API keys. The honest ones are easy to identify (they'll show specific projects, talk about model selection, discuss evaluation methodology). The marketing-driven ones produce impressive demos that don't reflect production-ready work. Hourly rates typically $200-$400. Project sizes vary widely.

Match your project to the right category before evaluating specific agencies. A simple lead-routing automation doesn't need a custom development agency. A high-volume document processing project doesn't need a workflow reseller. Sending the same brief to all four categories produces non-comparable quotes because they're solving different problems with different cost structures.

How to source qualified agencies

The two main sourcing channels — Google search and agency directories — both have problems. Most "automation agency" Google searches return content marketing pieces and pay-to-play directories that don't pre-qualify agencies meaningfully. Better sourcing channels exist if you know where to look.

1. SaaS partner directories (for category 2 agencies). If your project is HubSpot-centered, the HubSpot Solutions Directory lists vetted partners with case studies and tier ratings. Same for Salesforce AppExchange consulting partners, Shopify Partner Directory, Stripe Verified Partners. These directories pre-qualify agencies based on real project completions and platform expertise, not just self-attestation.

2. Operator referrals. Ask 3-5 operators in your network who they've used for automation work and what they thought. Operator-to-operator referrals are the highest-signal sourcing channel because the referrer has nothing to gain and you can ask probing questions. Even better: ask people who hired an agency and later replaced them — they'll tell you exactly what went wrong.

3. Industry-specific communities. Many industries have communities (Slack groups, Discord servers, private forums) where operators share vendor recommendations. The Pavilion community for B2B SaaS operators. The Operator community for early-stage startups. Industry-specific groups for verticals like home services, ecommerce, professional services. Recommendations in these communities are typically high-signal.

4. GitHub and technical communities. For technical automation projects, look at GitHub. Agencies that contribute to open-source automation tools, write technical blog posts, present at conferences — these have a public technical track record you can evaluate. Same for technical communities like Hacker News, dev.to, technical Reddit subs.

5. Vetted partner networks. Some independent organizations curate vetted automation partners. The quality varies, but some networks do real vetting (technical interviews, reference checks, project audits) and can save you significant pre-qualification time. Look for networks that publish their vetting methodology.

What to avoid: cold outbound from agencies (the best agencies don't cold-call), Clutch reviews (heavily gamed), random Google results for "best automation agency 2026" (these are content marketing plays).

The pre-call qualification (do this before any sales call)

Sales calls take 45-60 minutes per agency. If you take calls with 8 agencies, that's a full work day of sales calls. Pre-qualification cuts that significantly.

For each agency you're considering, spend 10-15 minutes verifying:

1. Company age and operating history. Check the agency's LinkedIn, registered business entity, and domain registration date. Real agencies have at least 3 years of operating history. Anything under 2 years is high-risk regardless of how impressive the marketing material looks. Use whois lookups to check domain age — agencies that registered their domain in the last 12 months should be treated as new entrants, not established firms.

2. Team size and named technical leadership. The "About" page should name specific people, especially technical leads. If everyone is "Director of Strategy" or "Head of Growth" and nobody is "Director of Engineering" or "Lead Architect," that's information. The agency might be sales-heavy and execution-light. Check the named technical people on LinkedIn — do they have automation engineering background, or are they marketers with engineering titles?

3. Specific case studies in your stack. Generic "we helped a client increase revenue by 40%" case studies tell you nothing. Specific case studies that name the systems, describe the architecture, and quantify the outcome are real evidence. Bonus signal: case studies that name the client (with permission), include direct client quotes, and link to the client's public site for verification.

4. Technical content output. Agencies that publish technical blog posts, contribute to open source, or speak at technical conferences have public artifacts of their technical depth. Marketing-driven agencies publish thought leadership about AI trends. Engineering-driven agencies publish technical posts about specific integration challenges. The difference is visible.

5. Pricing transparency. Some agencies publish pricing or pricing ranges on their site. Most don't. That's fine — pricing depends on project scope. But agencies that refuse to share even rate ranges in early conversations are usually pricing tactically based on what they think you'll pay, not based on cost. Tactical pricing is a yellow flag.

After pre-qualification, your initial list of 10-15 agencies should drop to 4-6 worth a real sales conversation. Better to take 5 thoughtful sales calls than 12 rushed ones.

The discovery call: questions that separate signal from noise

Discovery calls are where agencies signal their actual approach. Some agencies will spend the call asking you smart questions about your business. Others will spend it pitching their methodology. The asymmetry is information.

Twelve questions to ask in discovery calls:

1. Tell me about a project that went badly. What happened and what did you change? Every agency has bad projects. The agencies worth hiring have learned from theirs. Agencies that say "we don't have any bad projects" are either inexperienced or dishonest. Listen for specific learnings and process changes, not generic "we improved communication" answers.

2. Walk me through how you scope a project. Good answer: detailed multi-phase discovery process with named deliverables and clear scope boundaries. Bad answer: "we usually just have a few calls and write a proposal." The scoping process determines whether you'll have a clean project or a scope-creep nightmare.

3. Who specifically will work on my project? Names. Roles. Allocation percentages. If they can't give you names because "we haven't staffed it yet," ask who would likely staff it given the work. If they won't commit to specific staffing in the contract, that's information.

4. What's your code ownership and IP policy? Good agencies are clear: you own the code you paid for, with reasonable carve-outs for the agency's reusable framework code. Bad agencies are vague or insist on owning the work product. Get this in writing.

5. How do you handle scope changes mid-project? Good answer: a documented change-order process with explicit estimation, approval, and billing. Bad answer: "we're flexible" or "we just absorb minor changes." Vagueness here is a setup for either constant change-order fights or the agency absorbing scope and burning out on your project.

6. What's your testing and quality assurance process? Good answer: specific testing approach including unit tests where appropriate, end-to-end testing, user acceptance testing with you, and a staging environment for review. Bad answer: "we test thoroughly before launch." Vagueness here means the testing burden will fall on you.

7. What happens if I want to pause or cancel the project? Good agencies have clear policies — work completed gets billed, work in progress gets handed over, no penalty for pausing within reason. Bad agencies have aggressive cancellation clauses or refuse to discuss the topic.

8. Who is responsible after launch when something breaks? Good answer: a warranty period (typically 30-90 days) during which post-launch fixes are included, then a clear ongoing maintenance offering. Bad answer: "we make sure it works at launch, then it's on you." That answer means you're going to fight about post-launch issues.

9. What documentation will you provide? Good answer: technical architecture documentation, runbooks for operations, video walkthroughs for end users, and a knowledge handover session. Bad answer: "we document as we go." Documentation is one of the most-skipped deliverables and the one you most need at launch.

10. What's your client retention rate? The agencies worth working with retain most clients across multiple projects. If the agency does one project per client on average, that's information about quality. Ask for specifics — what percentage of clients return for second projects within 12 months?

11. Can I talk to 2-3 clients with similar projects to mine? Real agencies say yes immediately. Sales-heavy agencies hesitate or offer "case study calls" instead of actual references. References from similar projects (similar stack, similar size, similar industry) are far more valuable than random testimonials.

12. What would you not take on, even if I tried to pay you? Good agencies have boundaries — work outside their expertise, projects they can't scope cleanly, clients they can't serve well. Agencies that say "we can do anything" are either dishonest or have no boundaries, which means they'll take on projects they shouldn't.

Reference checks that actually work

Reference checks are usually theater — the agency gives you three happy clients, you have three pleasant calls, you learn nothing. Better reference checks are possible.

First, ask for references in your stack or industry. Generic references tell you the agency can deliver something. Stack-specific references tell you whether they can deliver what you specifically need.

Second, ask the agency for references from recent projects within the last 12 months. Old references can be from projects that pre-date current team or methodology. Recent references reflect the agency you'd actually hire.

Third, ask the agency for one reference where things went wrong. Every agency has them. The agencies worth working with can identify a project that didn't go smoothly and can describe what they learned. If they claim no project ever had problems, they're hiding something.

Fourth, on the actual reference calls, ask specific questions:

• "What was the original timeline and what was the actual timeline?"
• "What was the original budget and what was the final cost?"
• "What surprised you during the project that you wished you'd known upfront?"
• "How did they handle the most difficult moment in the project?"
• "What's the post-launch support been like?"
• "Would you hire them again? What would you do differently?"

The answers to these questions tell you what working with the agency actually looks like, not what their marketing says it looks like. The references that say "they were great, ahead of schedule, under budget, perfect" are usually rehearsed. The references that give you nuanced answers about what went well, what went poorly, and what they learned are the useful ones.

Fifth, find your own references. Look at the agency's LinkedIn — who are their connections in client-side roles? Reach out cold to a few clients the agency didn't cherry-pick for you. "Hey, I saw you worked with [Agency] on a project. We're considering them — can I ask you about your experience?" Most operators are willing to share honest assessments when asked directly.

Contract terms that protect you

The contract is the part most operators skip. Wrong move. The contract determines what happens when something goes wrong — and something usually goes wrong. Here are the terms that matter:

1. Milestone-based payment. Avoid agencies that want 100% upfront. Avoid agencies that want 50% upfront. Standard is 25-30% upfront, with the balance tied to specific milestones (discovery complete, build complete, UAT passed, launch). Milestone payments align incentives — the agency gets paid when work is delivered, not before.

2. Clear scope and out-of-scope definitions. The contract should include the brief (or a summary of it) and explicitly call out what's out of scope. Most contracts are deliberately vague to give the agency flexibility. Push for specificity — what gets built, what doesn't, what changes require a change order.

3. Code ownership and IP transfer. You should own the code you paid for. Standard language: "all custom work product delivered under this engagement is work-for-hire owned by [Client], with the exception of [Agency]'s pre-existing IP, reusable frameworks, and tools." Get this clear. Agencies that resist clear IP language are protecting future revenue at your expense.

4. Source code access and escrow. The agency should provide source code throughout the project, not just at launch. You should have your own copy of the repository. For larger projects, consider source code escrow — a neutral third party holds a copy that releases to you under specific conditions (agency goes out of business, agency stops responding).

5. Termination clauses. What happens if you want to cancel? Standard: you pay for work completed plus reasonable wind-down costs, agency hands over work in progress, no penalty fees. Bad: aggressive cancellation fees, work-in-progress withheld until full project value paid, NDAs that prevent you from describing the engagement.

6. Warranty period. Standard: 30-90 days after launch, the agency fixes bugs and issues in the original scope at no additional cost. Negotiate for 90 days minimum. Without a warranty period, every post-launch issue becomes a paid change request.

7. Liability and indemnification. The agency should indemnify you against claims arising from their work (e.g., if their code infringes someone's IP). Standard caps liability at the project value. Pay attention to mutual indemnification clauses — they should be balanced, not asymmetrically favoring the agency.

8. Data handling and security. If the automation touches sensitive data (customer PII, financial data, healthcare data), the contract should specify data handling requirements, breach notification, and compliance obligations. For regulated industries, this is non-negotiable.

Have a lawyer review the contract. A 2-hour contract review costs $400-$1,000 and routinely catches terms that would have cost you 10x that if invoked. Lawyers familiar with technology services contracts know what to look for.

What to do in the first 30 days of engagement

The first 30 days set the tone for the entire engagement. Get them right.

1. Insist on a kickoff with the actual team. Not just the account manager and the salesperson. The technical leads who will build the automation should be on the kickoff call. If they're not, that's information — the agency is using sales staff to manage the relationship rather than putting builders in front of the client. Push back. "I'd like to meet the technical leads who will be building this before we get into work" is a reasonable request.

2. Establish communication norms. Async channel for non-urgent communication (Slack, email, project management tool). Weekly status calls. Real-time channel for urgent issues. Clear escalation path if the agency goes silent. These aren't rigid rules — they're alignment about how the relationship works. Establishing them upfront prevents the "I emailed you three times" frustrations later.

3. Review the project plan in detail. The agency should produce a project plan within 1-2 weeks of kickoff. Review it carefully. Are the milestones tied to specific deliverables? Are the timelines realistic? Are dependencies identified? Are decision points called out? A vague project plan predicts a vague project. Push for specificity before any significant work happens.

4. Test their responsiveness early. Send a small clarifying question 24 hours after the kickoff call. See how fast and how thoughtfully they respond. If they take 4 days to answer a simple question early in the relationship, what will happen when they're deep in build work and you have an urgent issue?

5. Watch the first deliverable closely. The first significant deliverable (a discovery document, a technical architecture, a wireframe) is a quality signal. Is it specific or generic? Does it reflect your business or could it have been written for any client? Does it ask thoughtful questions or just restate what you said? The first deliverable is the agency's first opportunity to show you what they'll consistently produce.

If the first 30 days raise concerns — vague plans, slow responses, generic deliverables, communication gaps — address them immediately. Don't wait until the project is 60% complete to confront problems that were visible from week 2. Most failed agency engagements showed clear warning signs in the first 30 days that the client ignored, hoping things would improve. They rarely do.

If the first 30 days go well — clear plans, fast responses, specific deliverables, smooth communication — you probably picked the right agency. Trust the process and let them build.