Python: Chroma and Weaviate filters double apostrophes in string values, so values like O'Brien never match #14559
Description
What happens
A filter that compares a field to a string containing an apostrophe never matches in the Python Chroma and Weaviate connectors. The apostrophe is doubled (O'Brien becomes O''Brien) before the value is handed to the store, so it no longer equals the stored value.
Where
python/semantic_kernel/connectors/chroma.py ChromaCollection._lambda_parser, and python/semantic_kernel/connectors/weaviate.py WeaviateCollection._lambda_parser, on current main (64005c7). Both have this in the ast.Constant case:
case ast.Constant():
value = node.value
if isinstance(value, str):
return value.replace("'", "''")
if isinstance(value, bytes):
return value.decode("utf-8").replace("'", "''")That is SQL-style escaping. These two connectors do not build a SQL or query string. Chroma gets a where dict ({"name": value}, {"$and": [...]}) and Weaviate gets Filter.by_property(...).equal(value) objects, so the value is sent as data and nothing needs escaping. The other dict-based connectors (MongoDB, Pinecone, Qdrant) return node.value unchanged. Only the string-building connectors (Azure AI Search, Postgres, SQL Server, Oracle) need the doubling.
Steps to reproduce
Real Chroma, no mocks (pip install chromadb):
import asyncio
from dataclasses import dataclass
from typing import Annotated
import chromadb
from semantic_kernel.connectors.chroma import ChromaCollection
from semantic_kernel.data.vector import VectorStoreField, vectorstoremodel
@vectorstoremodel
@dataclass
class Rec:
id: Annotated[str, VectorStoreField("key")]
name: Annotated[str, VectorStoreField("data", is_indexed=True)]
vec: Annotated[list[float] | None, VectorStoreField("vector", dimensions=2)] = None
async def main():
col = ChromaCollection(record_type=Rec, collection_name="people", client=chromadb.EphemeralClient())
await col.ensure_collection_exists()
await col.upsert([Rec(id="1", name="O'Brien", vec=[0.1, 0.2]), Rec(id="2", name="Smith", vec=[0.2, 0.1])])
print(col._build_filter("lambda x: x.name == \"O'Brien\""))
for flt in ["lambda x: x.name == \"Smith\"", "lambda x: x.name == \"O'Brien\""]:
res = await col.search(vector=[0.1, 0.2], top=5, filter=flt)
print(flt, "->", [r.record.id async for r in res.results])
asyncio.run(main())Actual
{'name': "O''Brien"}
lambda x: x.name == "Smith" -> ['2']
lambda x: x.name == "O'Brien" -> []
The record with name="O'Brien" exists but is never returned. For Weaviate, _build_filter("lambda x: x.name == \"O'Brien\"") returns _FilterValue(value="O''Brien", operator=Equal,).
Expected
{'name': "O'Brien"}, and the search returns record 1.
I searched open and closed issues and PRs for this and found nothing. I am opening a PR right after filing this.
Activity
Python: Stop doubling apostrophes in Chroma and Weaviate filter values (