Building an Autonomous Job Application Agent

The Problem with Job Applications

Software engineers applying for senior roles typically apply to 30–60 positions per cycle. Each application involves the same loop: search across boards, read JD, decide if it fits, tailor resume, write cover letter, navigate an ATS, upload files, fill forms. The cognitive load is enormous and most of it is mechanical.

The interesting question is not "can AI help?" — it can. The interesting question is where the human judgment boundary should sit. Sending applications without any human review is dangerous: a hallucinated credential or a misread seniority level can damage relationships. Reading every application before submission is the right model, but it should take minutes, not hours.

This project builds a four-agent pipeline that does all the mechanical work and pauses at a clearly-defined review gate. By the time you see something, it has already been sourced, scored, tailored, and packaged — you just decide whether to send it.

---

Project Overview

Target Users: Software engineers, data scientists, and technical professionals running structured job searches.

The Four Agents:

Pipeline State Machine:

IDLE → SCOUTING → FILTERING → TAILORING → PENDING_APPROVAL → SUBMITTING → COMPLETE

The pipeline pauses at PENDING_APPROVAL. Nothing gets submitted without an explicit approve action.

---

System Architecture

---

Agent Workflow

---

Technical Deep Dive

Agent 1 — Scout

The Scout builds search queries from the user's SearchPreferences (inferred by cv_parser: titles like "Senior SWE", "Staff Engineer" based on experience) and fetches HTML from 5 boards: LinkedIn, Indeed, Glassdoor, Greenhouse, and Lever.

Board HTML is not structured data. It's rendered React apps, server-side templates, or dynamic JSON embedded in script tags. Rather than writing and maintaining 5 board-specific parsers, Gemini extracts structured job listings from whatever HTML it receives. The system prompt enforces one strict rule: only return URLs that literally appear in the HTML. Gemini cannot fabricate a posting it did not see.

After extraction, url_validator runs concurrent HTTP HEAD requests (5 concurrent, 3s timeout) and checks body patterns for "position filled," "listing expired," or similar signals. Any posting older than 30 days or returning non-2xx is dropped. The Scout returns only verified, fresh listings.

BrightData Web Unlocker handles boards that block direct requests. The fallback pattern: try direct HTTPX first, fall back to BrightData on 403 or CAPTCHA detection.

Agent 2 — Matcher

The Matcher scores each job description against the user's resume using a four-factor weighting:

Gemini returns llm_score (0–100), a list of missing_skills, hard_constraints, and a boolean proceed flag. Jobs below the threshold (default 75) are dropped.

The pgvector infrastructure is present but the Matcher currently runs on LLM scoring only. Vector cosine similarity between the resume embedding and JD embedding is wired as vector_score = 0.0 — a deliberate placeholder. The scaffolding is ready for semantic augmentation; we chose not to activate it in v1 to avoid the cost of embedding every JD.

Agent 3 — Tailor

The Tailor is the most carefully constrained agent in the pipeline. Its system prompt contains an explicit anti-fabrication rule: resume tailoring may only reorder or rephrase existing achievements, never add skills, experiences, or accomplishments the user does not have.

This constraint is not just ethical — it's practical. A fabricated credential that survives to the phone screen will be immediately exposed. The agent's value is emphasis and framing, not invention.

The cover letter prompt has a parallel constraint: it may only reference "achievements from the resume." No generic statements about the company culture, no made-up passion for the role. Every sentence must be traceable to something in the tailored resume JSON.

PDF generation uses Jinja2 to convert the resume JSON to styled HTML, then pdfkit (wkhtmltopdf wrapper) for PDF conversion. The fallback saves HTML directly if wkhtmltopdf is not installed — useful in development environments.

Agent 4 — Submitter

The Submitter is the highest-risk agent and has the most defensive architecture.

It detects ATS type from the URL and HTML: Greenhouse URLs contain /jobs/ with numeric IDs, Lever contains /apply/. Detection feeds the fallback selector maps — hardcoded CSS selectors for first_name, email, resume_upload, and submit per ATS platform.

The primary path uses Gemini to analyze the form DOM and generate an ordered action sequence:

[
  { "action": "FILL", "selector": "#first_name", "value": "Santanu" },
  { "action": "FILL", "selector": "#email", "value": "user@example.com" },
  { "action": "UPLOAD", "selector": "input[type=file]", "value": "/path/to/resume.pdf" },
  { "action": "CLICK", "selector": "button[type=submit]" }
]

Playwright Stealth executes these actions with human-like timing: 40–120ms per character, 1–3.5s pauses between actions, Bezier curve mouse movement, spoofed navigator.webdriver, and a Chrome 124 Windows 10 user agent. Every session saves a Playwright trace zip to logs/traces/ and a screenshot on error.

---

Workflow Execution Example

User uploads their CV and triggers the pipeline for "Senior ML Engineer, Remote, $180k+".

---

Challenges & Solutions

The CAPTCHA handling deserves emphasis. Rather than attempting CAPTCHA solving (which is both fragile and legally ambiguous), the Submitter returns CAPTCHA_FAILED and the user submits that specific application manually. The pipeline continues with the remaining packets. Graceful degradation over brittle automation.

---

Observability

The workflow_trace — an ordered array of TraceStep objects — is stored in ApplicationHistory.workflow_trace_json. Every agent and tool logs: who acted, what action, a summary, timestamp, and optional payload snapshot.

The Google ADK web UI (adk web .) provides an interactive trace viewer during development, showing the reasoning graph and letting you replay any analysis step by step.

Key metrics to watch in production:

• Match score distribution by board (is LinkedIn consistently scoring higher?)
• Tailor → Approve rate (are users accepting the tailored docs?)
• Submitter success vs. CAPTCHA rate per ATS platform
• Scout URL validation drop rate (high drop = boards changing structure)

---

Security & Privacy

Resume data: Stored in PostgreSQL user_profiles table with user_id as primary key. The base_resume_json column is structured JSON — not encrypted at rest in v1, but the schema is designed for column-level encryption.

Gemini calls: Resume text and JD content are sent to Gemini API. No API-level logging of prompt content beyond what Google's standard retention policy includes. Users should be aware their resume content is processed by an external LLM.

Browser automation: Playwright runs headless with all standard automation signals spoofed. The target sites (Greenhouse, Lever) permit automated API access in their developer terms; the stealth measures protect against aggressive rate limiting, not circumvention of explicit prohibitions.

---

Future Enhancements

Phase 1 — Smarter Sourcing

• Activate pgvector scoring to supplement LLM match scoring
• Add Greenhouse and Lever direct API access (both have public APIs) to reduce scraping fragility
• Referral network lookup: identify second-degree LinkedIn connections at target companies

Phase 2 — Richer Tailoring

• ATS keyword gap analysis: compare tailored resume against JD keyword frequency before submitting
• Multiple resume variants: engineer, manager, IC-track versions auto-selected by JD seniority signals
• Salary negotiation brief generation triggered at offer stage

Phase 3 — Full Campaign

• Follow-up email drafting on day 5 post-submission
• Interview prep: role-specific Q&A generated from JD + resume
• Offer comparison model with total compensation calculator

---

Key Learnings

The HITL gate is the product. Users did not trust the system to submit without review, and that trust instinct is correct. The gate is not a limitation — it is what makes the system safe to use. Everything before the gate runs autonomously; the gate is the human's moment. Design around it, not against it.

Scraping is fragile; validator is essential. The URL validator — HTTP HEAD checks + body pattern scanning + date recency filter — runs after every Scout cycle. Without it, ~30% of results would be expired, filled, or returning 404. The validator is not infrastructure overhead; it is a core agent function.

LLM scoring beats keyword matching for senior roles. Senior roles and staff-level positions have JDs with similar keywords. Vector similarity and keyword overlap fail to distinguish "requires 8+ years of distributed systems" from "nice to have experience with distributed systems." Gemini's LLM scoring, weighted toward experience-level alignment, produces better discrimination for seniority-sensitive roles.

Playwright traces save hours of debugging. The Playwright trace zip is generated for every submission attempt. When a form submission failed on a Lever ATS variant, the trace showed exactly which selector produced a stale reference. Without traces, debugging ATS-specific failures is guesswork.

---

Conclusion

The Job Scout Agent collapses the mechanical overhead of a job search into a pipeline with a single human decision point: approve or reject each tailored application package. The four agents handle discovery, evaluation, customization, and submission — but they pause before anything matters.

The technical architecture reflects a clear philosophy: deterministic tools for facts (URL validation, PDF generation), LLM for judgment (scoring, tailoring, form analysis), and human oversight for consequential irreversible actions (sending applications). That division produces a system that is both autonomous enough to be useful and bounded enough to be trustworthy.