Aller au contenu
login
arrow_backRetour aux issues
semantica-agi/semantica #997

perf(semantic_extract): cache spaCy model loading in NERExtractor

ecoDébutant enhancement help wanted performance

descriptionDescription

## Problem `NERExtractor` can load the spaCy model directly with `spacy.load()` during initialization instead of using the model-loading cache already available in `semantica.semantic_extract.methods`. This means repeated construction of `NERExtractor` instances can repeatedly load the same spaCy model from disk, adding significant startup/processing overhead. ## Current behavior The `NERExtractor` initialization path uses a direct spaCy load when the `ml` extraction method is selected: ```python spacy.load(self.model_name) ```` This bypasses the cached model-loading path introduced/used in `semantic_extract.methods`. As a result, callers that repeatedly create `NERExtractor` instances can repeatedly incur the cost of loading the same model. ## Why this matters spaCy model loading is substantially more expensive than processing an already-loaded document. Repeated model initialization can therefore: * increase request latency; * increase CPU and disk I/O; * increase memory churn; * become particularly expensive for batch processing; * unnecessarily duplicate model instances in long-running applications. This is especially relevant because `NERExtractor` is a higher-level extraction API and can be used independently of the Explorer endpoint. ## Suggested fix Reuse the existing centralized spaCy model-loading/cache mechanism from `semantica.semantic_extract.methods` instead of calling `spacy.load()` directly. The fix should preserve the existing model selection and error/fallback behavior. Please also add a regression test that verifies repeated `NERExtractor` usage does not repeatedly call `spacy.load()` for the same model. ## Scope This is intentionally separate from PR #886. PR #886 fixes the specific uncached extraction functions associated with #889, while this issue covers the remaining `NERExtractor` loading path.
codeOuvre sur GitHub