APACHE-2.0
OPEN-SOURCE STORAGE
WHY WE BUILT THELAKE
01
Built for full payloads
Traces land in Zstd-compressed Parquet sorted by session_id and timestamp on object storage, keeping 365-day capture free on Softprobe Cloud.
02
Fast session listing
A rebuildable Postgres session_summary directory powers search and pagination without scanning raw Parquet files on every page load.
03
Open format, your files
Read the Rust code on GitHub, query traces directly with DuckDB SQL or Grafana Loki/Tempo routes, or run thelake inside your own VPC.
BUILT FOR ENGINEERS
-- Query traces directly via make duckdb-shell (DuckLake) SELECT session_id, COUNT(*) AS charge_calls, MAX(http_response_status_code) AS last_status FROM traces WHERE http_request_method = 'POST' AND http_request_path = '/api/charges' AND timestamp >= NOW() - INTERVAL '7 days' AND session_id <> '' GROUP BY session_id HAVING COUNT(*) > 1 ORDER BY charge_calls DESC LIMIT 25;
STORAGE ARCHITECTURE
Storage layer
Hot SSD and indexed search clusters
Zstd-compressed Parquet on S3, GCS, or local disk
Large payloads
Sampled out or truncated to protect cluster cost
Full prompts, tool calls, and HTTP bodies kept intact
Session listing
Aggregates raw spans across the time window
Rebuildable Postgres directory (session_summary)
Query & schema
Proprietary query syntax and fixed index fields
DuckDB SQL, MAP attributes, and typed column promotion
Retention & price
Metered per gigabyte; typically capped at 14 to 30 days
365 days free on Softprobe Cloud, or self-hosted in your VPC
Written in Rust on Apache Arrow, DuckDB, and DuckLake (Apache-2.0).
DEPLOYMENT MODELS
Softprobe Cloud
$0 · FREE FOREVER
Hosted thelake storage wired straight into Softprobe Explorer, Fix Briefs, and Softprobe MCP.
✓
Unlimited session and backend HTTP capture
✓
365-day session history included
✓
1-prompt setup (OpenCode, LangChain, Vercel)
✓
Instant Explorer access with nothing to manage
Enterprise BYOC
YOUR OWN VPC
Run thelake in your own AWS, GCP, Azure, or EU cloud while using Softprobe Explorer and QA checks.
✓
Parquet files stay in your own S3 or GCS buckets
✓
Prompts, tool calls, and HTTP bodies never leave your VPC
✓
Query DuckLake directly with DuckDB inside your network
✓
Custom retention, SSO/RBAC, and enterprise SLAs
APACHE-2.0 OPEN SOURCE