Athena Blue

The project

Corporates that impact and rely upon coastal environments should be able to find and fund restoration projects that protect and regenerate that specific environment, and the projects should be able to prove they’re worth funding.

I joined to help them use current AI tooling to build a scalable process while figuring out the market dynamics

The problem

Projects were assessed across a range of criteria from impact to governance, with our Subject Matter Experts (SMEs) spending at least 3 hours pulling answers from long, unstructured project documents. Those paying for the assessment rely upon the output for investment decisions, so the output must be accurate and defensible.

The challenge here was, how do you produce something consistent and reliable with the aid if probabilistic models?

These are the actions I took to drive our product offering towards scalability and growth.

Rebuild the workflow around the model, not the model around the workflow

The existing 150 question form was designed for a human expert to work linearly, and most questions were only pertinent for some projects. Rather than point the LLM at the existing form and have it autofill, I sat with the SMEs and captured the nuance they were following in their heads: which questions depended on which, what triggered a deeper dive, and what could be skipped. That became a modelled workflow with gating rules, then interactive prototypes, then a build.

The result was an adaptive, tiered intake. Answers are produced with citations for approval, and route you to the next relevant tier rather than the next row of the form. Users saw around 30% fewer questions. Completion time went from over three hours to 20-30 minutes, when paired with the effects of the following steps.

Make the engine trustworthy before expanding the reach

To begin with the extraction engine was getting around half of its answers right. The temptation at pre-seed is to widen scope and show more, but I felt the effort was better spent on designing the evals and feedback loops that improve accuracy with every iteration. This is how I went about it

• Correction capture in the product: A user is presented with an extracted answer along with a citation and confidence score. When they correct an extracted answer, that correction is stored as a ground truth, against the original answer and the source document it came from. We’re harnessing the user’s own actions as eval labelling.
• An audit trail data model: Every answer carries a trail of where it came from, what the user said and what the engine said. It builds trust and became the foundation for everything else.
• Threshold gated guidance updates: Our LLMS rely upon context and guidance for the answer extractions. For the guidance to be updated the corrections must show the same failure across multiple projects and be signed off by a human. Once the engine guidance is updated I run a ground truth function to compare the new output accuracy to a previous run across labelled a labelled corpus - after all it is only probabilistic.
• Passing threshold defined per field: Not every answer needs the same accuracy. I set the bar field by field and traded accuracy against cost and latency.

Accuracy went from under 50% to around 80%, meaning the user only has to correct or complete 20% of the answers. Remember they’re already only presented on average 70% of the full set, so they’re now completing 20% of 105 questions rather than 50% of 150.

Sell to the demand side first

Marketplace startups have a catch 22 cold start problem: projects won’t list without buyers, buyers won’t come without projects. Rather than build both sides of the marketplace, I leveraged our unique position and sequenced it.

Rather than building out the entire vertical assessment step as a starting point, we sold assessment and exposure services to paying demand-side customers first and linked them to projects we’d already onboarded. We onboarded a handful of early projects for free to build supply and facilitate liquidity.

We looked at white-labelling but decided against it as the burden of a requisite full build against our terms didn’t match the upside.

Shipped

• Adaptive, tiered assessment intake with conditional routing
• Extraction engine improvement loop: correction capture, audit trail, gated guidance updates
• Product analytics from scratch: funnels, drop-off, confusion signals, time-on-task to evidence time saved
• AI assisted UX testing: an LLM roleplaying target user profiles, paired with expert review, finding friction in days rather than waiting on scheduled testing rounds
• Working prototypes and build ready specs in Claude Code, implemented directly by the engineers
• Data handling framework and client facing documentation for a client base sceptical of AI handling their data

On the commercial side I onboarded the first ten customers myself and kept those relationships after the sale. The feedback I received, paired with the analytics, is what steered the roadmap. The early prototypes went into investor demos and helped secure the initial funding.

What I’d do differently

I should have killed the vertical build sooner. With stakeholder pressure geared towards securing funding through an end to end process, I spent the first couple of months geared towards this. But the business is the marketplace, and this is where we could most effectively harness the founder’s deep expertise and network.

If I was starting again I would cut away the fat to build the leanest process that allowed both buyers and sellers to onboard. With that extra bit of runway, we would have had more demand side validation before the funding raise and a clearer signal on pricing models.