Aller au contenu
login
arrow_backRetour aux issues
adhabnr-ux/evalport #18

Retrofit 21 adapters + 2 reference adapters to self-referencing pinned extras (tracking issue for Discussion #13)

ecoDébutant help wanted good first issue

descriptionDescription

Tracking issue for the still-open "question 2" from [Discussion #13](https://github.com/adhabnr-ux/evalport/discussions/13#discussioncomment-18102622): `CONTRIBUTING.md` now documents the **self-referencing** extras form for new adapters — ```toml [project.optional-dependencies] yourframework = ["yourframework-package>=X.Y.Z"] test = ["pytest", "your-adapter-name[yourframework]"] ``` — instead of duplicating the version string in both `test` and the named extra. This was proven to work (`pip install -e ".[test]"` resolves recursive self-extras correctly in modern pip) and applied to the newest adapter (`opencompass-openeval-adapter`) when the convention was written down, but the other 21 adapters plus the 2 reference adapters (`autogen-openeval-adapter`, `crewai-openeval-adapter` — the ones `CONTRIBUTING.md` itself points new contributors at) still use the older duplicated-string form. **Why this is a good first issue, not just busywork:** the duplicated form isn't broken, but it's the exact shape that let a real bug through once already — see the `giskard-openeval-adapter` writeup in [Discussion #13](https://github.com/adhabnr-ux/evalport/discussions/13#discussioncomment-18088432), where an unpinned/duplicated constraint let a breaking upstream API change go undetected until someone happened to test against the newer version by hand. The fix here is small and mechanical per adapter (edit one `pyproject.toml`, re-verify `pip install -e ".[test]"` + `pytest` still passes in a fresh venv), so it's a safe, well-scoped way to touch every adapter package in the repo without needing framework-specific expertise going in. **What to do, per adapter** (all 23 listed below): 1. Change `test = ["pytest", ..., ">=X.Y.Z", ...]` to `test = ["pytest", ..., "[]"]`, removing the duplicated version string. 2. In a fresh venv: `pip install -e ".[test]"` then `pytest tests/ -v` — confirm the same test count passes as before (no regressions from the extras change alone). 3. Check off the adapter below in your PR description. **Adapters to convert (23):** - [ ] `autogen-openeval-adapter` (reference adapter) - [ ] `crewai-openeval-adapter` (reference adapter) - [ ] `ragas-openeval-adapter` - [ ] `langsmith-openeval-adapter` - [ ] `braintrust-openeval-adapter` - [ ] `mlflow-openeval-adapter` - [ ] `opik-openeval-adapter` - [ ] `phoenix-openeval-adapter` - [ ] `weave-openeval-adapter` - [ ] `uptrain-openeval-adapter` - [ ] `langfuse-openeval-adapter` - [ ] `giskard-openeval-adapter` - [ ] `llamaindex-openeval-adapter` - [ ] `patronus-openeval-adapter` - [ ] `vertexai-openeval-adapter` - [ ] `dspy-openeval-adapter` - [ ] `haystack-openeval-adapter` - [ ] `evidently-openeval-adapter` - [ ] `guardrails-openeval-adapter` - [ ] `argilla-openeval-adapter` - [ ] `azure-ai-evaluation-openeval-adapter` - [ ] `arthur-bench-openeval-adapter` - [ ] `athina-openeval-adapter` (`huggingface-evaluate-openeval-adapter`, `lm-eval-harness-openeval-adapter`, `lighteval-openeval-adapter`, `opencompass-openeval-adapter`, `financebench-openeval-adapter`, `deepeval-openeval-adapter`, and `galileo-openeval-adapter` are excluded — either already self-referencing, or a static-dataset target with no version to pin per `CONTRIBUTING.md`'s documented exception.) You don't have to do all 23 in one PR — a PR covering a handful at a time is completely fine, and probably easier to review. Comment here if you want to split it up or claim a subset.
codeOuvre sur GitHub