llm A field note by Vikrant Sharma
LLM gateways are becoming PII firewalls
Consus launched on HN today as a proxy layer between your app and OpenAI that strips PII before the prompt leaves your network. The compliance angle is obvious but the redaction challenge is not.
A YC company called Consus launched on Hacker News today with a pitch that sounds obvious once you hear it: sit between your application and the LLM API, detect PII in outbound prompts, redact it, send the clean version to OpenAI, then put the real data back into the response before your user sees it. The compliance story writes itself. Healthcare apps cannot send patient names to third-party APIs without a BAA. Financial services cannot log credit card numbers in LLM request traces. A gateway that redacts before the data crosses the network boundary solves the paper trail problem without changing your application code. What interests me is the detection part. PII is not just email addresses and phone numbers. It is also the inference you can make from a prompt like “my patient in room 3B with the broken femur”. Consus claims to handle context-dependent redaction, which means they are running their own model to classify spans of text before deciding what to mask. That creates a performance trade-off. Every prompt now has two LLM calls: one to detect PII, one to answer the actual question. Latency doubles unless the detection model is fast and local. The HN comments mention this. One person asked about accuracy. Another pointed out that regex and named entity recognition have been solving parts of this problem for years without the API tax. The harder question is what happens when the PII is load-bearing. If you redact the patient identifier and the LLM returns generic advice, putting the name back in does not make the response useful. The gateway cannot fix a prompt that needed the sensitive context to work. Still, the category makes sense. Every company building LLM features is writing some version of this sanitisation layer in-house. Consus is betting that most teams would rather pay for a tested implementation than debug their own regex at 2am after a compliance audit. I would want to see the benchmark on false positives. If the gateway flags normal business terms as PII and blocks legitimate prompts, adoption dies fast.
Source: Launch HN: Consus (YC F24) – LLM gateway for companies handling sensitive data
