filter-semantic

You can semantically filter a collection by using an LLM evaluation callback to match items against a natural-language predicate. This goes beyond exact keyword matching to evaluate the descriptive meaning of each item.

Yes, you can exclude items by using the 'exclude' filtering mode, which returns non-matching items from the collection. This allows you to remove unwanted categories from your dataset based on a descriptive natural-language predicate.

Performing semantic filtering requires an LLM generation callback to evaluate items and a resource manager to handle collection operations. These prerequisites enable the system to process and refine your datasets based on natural-language predicates.

Natural-language predicate filtering works by passing a descriptive condition, such as 'discuss ethical implications', to an LLM. The LLM evaluates each note in the collection to isolate specific information and categorize notes based on semantic meaning.

Use LLM-based semantic filtering when you need to refine search results or categorize items based on conceptual meaning, such as finding research papers that are 'purely theoretical'. Standard keyword matching is insufficient for evaluating these nuanced, descriptive predicates.