arrow_backRetour aux issues
prometheus/prometheus
#14926
Débutant
Ouvrirarrow_forward
Débutant
Ouvrirarrow_forward
Débutant
Ouvrirarrow_forward
feat: Avoid decay of our micro-benchmarks; test them on CI
ecoDébutant
verifiedCNCF
help wanted
priority/P2
component/tests
area/ci-cd
descriptionDescription
### Proposal
As per https://github.com/prometheus/prometheus/pull/14925, our code tends to change, but microbenchmarks (`func BenchmarkXYZ(b *testing.B)`) decay. They always build, but the they decay "semantically" by essentially have a broken flows (timeouts, deadlocks), they code they benchmark is suddenly wrongly configured (e.g. so you don't benchmark success case anymore but failures/incorrect execution).
Microbenchmarks decay because (currently) we don't require to re-run the benchmarks on every change of the related code.
1) For this reason in the past I wrote for my "Efficient Go" book a quick [`testorbench` structure](https://github.com/efficientgo/core/blob/main/testutil/testorbench.go#L26) that allows running benchmarks "as a test with a N == 1" just to test runnability. `testorbench` also allows doing more tests on `!tb.IsBenchmark()`, as you don't want to do detailed asserts during benchmark time as it's waste of time (success criteria should be deterministic) and you want to measure execution not tests. You can see example use [here](https://github.com/bwplotka/benchmarks/blob/main/benchmarks/metrics-streaming/benchmark_test.go#L38).
2) One alternative would be to have some `go test -run ^$ -bench .* -benchtime 1x` just to at least see if things are runnable on CI.
3) Finally, if we get the nightly microbenchmark compared to e.g. day before build setup, that would keep the whole pipeline fresh perhaps and give us meaningful data. Maybe that's a solution too (: See [how Vitess is doing this](https://benchmark.vitess.io/).
## Acceptance Criteria
* Our micro-benchmark code is not decaying e.g. it's semantics and flow is updated when the related code is updated.
cc @bboreham @cstyan
Honestly (3) would be nice to do!
Issues similaires
medik8s/fence-agents-remediation
star18
Poids du dépôt moyen
v0.8.0 starts a cluster-scoped Secret informer despite namespace-scoped Secret RBAC
## Description While testing Fence Agents Remediation (FAR) v0.8.0, I observed the manager repeatedly attempting to li…
Go
good first issue
Wayfare-labs/wayfare
star1
Poids du dépôt léger
Check that an issuer's home_domain round-trips to the same stellar.toml
Written to the standard set by #33 — read that issue first for the expected level of detail and the acceptance criteria…
Go
help wanted
area:corridor
Wayfare-labs/wayfare
star1
Poids du dépôt léger
Report auth_immutable: whether an issuer's flags can still change
> **This issue is the reference standard.** Every other check issue links back here for the expected level of detail. I…
Go
help wanted
good first issue