Collect BanyanDB data files on e2e test failure#13765
Open
Collect BanyanDB data files on e2e test failure#13765
Conversation
- Update infra-e2e to 36221fa (adds collect-on-failure feature) - Add cleanup.collect to all 18 BanyanDB e2e YAMLs - Compose tests: collect by service name - Kind tests: collect by label-selector app=banyandb - Data dirs: trace, stream, measure, property, schema-property - Output to $SW_INFRA_E2E_LOG_DIR/banyandb-data (included in existing upload) TESTING: intentional failure in storage/banyandb to verify data collection
wu-sheng
added
the
test
Artifact confirmed: banyandb-data/ directory uploaded with trace (796K), property (1.1M), schema-property (14M), 234 files.
wu-sheng
changed the title
- Compose: storage/banyandb — failure added as LAST verify case (after all 122+ real cases run, ensuring all 5 data dirs populated) - Kind: profiling/ebpf/oncpu/banyandb — failure after profiling cases Both tests will fail late, with full data in trace, stream, measure, property, and schema-property directories.
Kind BanyanDB pods are in istio-system namespace (not default). Label selector updated to app.kubernetes.io/name=banyandb to match the SkyWalking Helm chart convention.
Use a real query (service layer) with wrong expected file instead of echo. This ensures real verify cases pass first (populating all 5 BanyanDB data dirs), then the intentional mismatch triggers failure after data is fully flushed across all retries.
Compose (storage/banyandb): trace 3.1M/926 files, property 1.1M, schema-property 14M Kind (ebpf/oncpu/banyandb): stream 256K/75 files, measure 20K/8 files, property 1.1M, schema-property 14M Both paths confirmed: data collected before cleanup and included in existing log artifact upload.
Member
Author
Verification ResultsBoth Docker Compose and Kind (K8s) data collection verified with intentional test failures. Docker Compose (
|
Base docker-compose overrides root paths: --stream-root-path /tmp/stream-data --measure-root-path /tmp/measure-data Collect from /tmp/stream-data/ and /tmp/measure-data/ instead of /tmp/stream/ and /tmp/measure/ which are empty.
Remove --stream-root-path and --measure-root-path from base-compose, storage, tls, and data-generate docker-compose files. All modules now use default /tmp root: /tmp/trace/, /tmp/stream/, /tmp/measure/, /tmp/property/, /tmp/schema-property/ Updated stages data-generate workflow and volume mounts to match. Collect paths in e2e YAMLs now align with actual data locations.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Collect BanyanDB data files on e2e test failure
Problem: When BanyanDB-related e2e tests fail, only container logs are uploaded. The actual
BanyanDB data files (trace, stream, measure, property, schema-property) are lost with container
cleanup, making it hard to diagnose storage-level issues.
Changes:
1. Collect BanyanDB data on failure
skywalking-infra-e2eto36221fa(addscollect-on-failure)cleanup.collectto all 18 BanyanDB e2e YAMLs:service: banyandbnamespace: istio-system+label-selector: app.kubernetes.io/name=banyandb$SW_INFRA_E2E_LOG_DIR/banyandb-data— included in existing artifact upload2. Remove custom root-path overrides from BanyanDB base compose
--stream-root-path /tmp/stream-data --measure-root-path /tmp/measure-datafrom:test/e2e-v2/script/docker-compose/base-compose.yml(affects all Compose BanyanDB tests)test/e2e-v2/cases/storage/banyandb/docker-compose.ymltest/e2e-v2/cases/storage/banyandb/tls/docker-compose.ymltest/e2e-v2/cases/storage/banyandb/data-generate/docker-compose.yml/tmproot path consistently:/tmp/trace/,/tmp/stream/,/tmp/measure/,/tmp/property/,/tmp/schema-property/Collected directories:
/tmp/trace/— trace data/tmp/stream/— stream data (logs, sampled traces)/tmp/measure/— measure data (metrics from OAL)/tmp/property/— property data/tmp/schema-property/— schema property dataVerified with intentional failures on both Docker Compose and Kind paths. See PR comments for
detailed artifact structure and data sizes.
CHANGESlog.