Hi HN, I built Spring AI Privacy Guardrails. I wanted tools to be able to use personal data without sending it to the LLM.
For example, a customer lookup request might include a customer ID and a phone number. Both are replaced with placeholders before the model sees the request. The lookup tool can get the real customer ID if the user's permissions and the tool's disclosure policy allow it. The phone number stays as a placeholder. Any PII detected in the tool's response is replaced before it goes back to the model.
For PII detection, you can use local rules, Presidio or OpenNLP. Spring Security filters which tools the model can see and checks permission again before execution.
There's also content inspection using rules, local ONNX models or HTTP guard models. It checks inputs and can check the final response too. With privacy protection enabled, detected PII is replaced before content is sent to an external guard model.
You can use the features separately or together in a Spring AI ChatClient.
The project was mentioned in the Spring blog's This Week in Spring and is listed in Awesome Spring AI.
How are you handling cases where a tool needs personal data that the LLM shouldn't see?
Hi HN, I built Spring AI Privacy Guardrails. I wanted tools to be able to use personal data without sending it to the LLM.
For example, a customer lookup request might include a customer ID and a phone number. Both are replaced with placeholders before the model sees the request. The lookup tool can get the real customer ID if the user's permissions and the tool's disclosure policy allow it. The phone number stays as a placeholder. Any PII detected in the tool's response is replaced before it goes back to the model.
For PII detection, you can use local rules, Presidio or OpenNLP. Spring Security filters which tools the model can see and checks permission again before execution.
There's also content inspection using rules, local ONNX models or HTTP guard models. It checks inputs and can check the final response too. With privacy protection enabled, detected PII is replaced before content is sent to an external guard model.
You can use the features separately or together in a Spring AI ChatClient.
The project was mentioned in the Spring blog's This Week in Spring and is listed in Awesome Spring AI.
How are you handling cases where a tool needs personal data that the LLM shouldn't see?