Trustworthy AI for software engineering · software security · reliable agent systems
I am a software engineer in Dublin building and studying AI systems that modify code or take external actions. My work asks a practical research question: what evidence is strong enough to trust an agent's output? I answer it with executable ground truth, failure-oriented experiments, explicit uncertainty, and reproducible artifacts.
Research portfolio · LinkedIn · Email
An execution-grounded audit of whether security checkers respond to program behaviour or merely to surface form.
- Controlled study: 5 oracles × 96 Python implementations across 8 weakness classes; all labels earned by running exploit witnesses.
- Production study: 140 real CVE fixes from 65 projects. The evaluated tools produced unchanged verdicts across 92.9%–98.6% of fixes.
- Cross-study result: synthetic and production detection rates correlate at Spearman ρ = 0.782.
- Reproducible CLI, fixed protocol, raw results, bootstrap intervals, production manifest, and green CI.
Status: public research artifact and manuscript; not peer-reviewed.
A controlled study of whether identical secure-coding guidance works differently in task prompts, repository instructions, both, or neither. The repository separates pilot tasks from a protected holdout, records a deterministic run schedule, and pre-specifies task-clustered analysis.
Status: work in progress; pilot evidence is non-confirmatory.
| Project | Problem | Evidence |
|---|---|---|
| PatchPilot | Provider-authored OpenAPI changes should migrate affected customer code instead of becoming unread changelog entries. Built in response to YC's Self-Maintaining APIs request. | Customers atomically grant, inspect, revoke, and re-grant provider-key access without erasing approval history; every campaign is authorized before token issuance, then the customer verifies its pinned key before deterministic JS/TS/Python edits become a PR. 127 tests, 16 releases, green CI. |
| Effect Broker | A timed-out agent action may have committed even when its acknowledgement was lost. | Stable idempotency contracts, authoritative reconciliation, outcome_unknown, and a process-kill crash matrix on SQLite and PostgreSQL. |
| Agent Redteam | Agent-security claims need outcome evidence, not subjective transcript review. | Executable attack oracles, clean-twin counterfactual replay, guardrail-effectiveness measurement, SARIF, and regression baselines. |
| Agentic Digital Twin | A public AI profile must not turn web search or model fluency into invented personal claims. | Source-grounded answers, a pre-prompt authority gate, measured retrieval and claim verification, and 216 offline tests. |
- Earn labels through execution. If a claim can be checked by running an exploit, crash, or contract test, run it.
- Model ambiguity explicitly.
UNKNOWNis a valid result; silence or timeout is not evidence of safety. - Make authority structural. External effects cross explicit, recorded approval boundaries.
- Publish the audit trail. Protocols, raw records, limitations, hashes, and negative results belong beside the code.
- Keep claims narrower than evidence. Artifacts are not publications, pilots are not confirmatory studies, and passing tests are not user adoption.
I hold an MSc in Cybersecurity from Dublin Business School (2025) and a bachelor's degree in Computer Science from Savitribai Phule Pune University (2024). I am preparing for PhD applications focused on trustworthy AI for software engineering, especially evaluation integrity, secure code generation, and dependable agent actions.
I welcome technically specific feedback, independent reproductions, and research collaboration. The best starting point is the oracle audit or an email to prathemesh7744@gmail.com.