"use strict"; const {Fragment: _Fragment, jsx: _jsx, jsxs: _jsxs} = arguments[0]; const {useMDXComponents: _provideComponents} = arguments[0]; function _createMdxContent(props) { const _components = { p: "p", ..._provideComponents(), ...props.components }, {Heading, Mermaid} = _components; if (!Heading) _missingMdxReference("Heading", true); if (!Mermaid) _missingMdxReference("Mermaid", true); return _jsxs(_Fragment, { children: [_jsx(_components.p, { children: "Understanding tif1’s data flow helps you optimize performance and troubleshoot issues. This guide explains how data moves from the CDN to your analysis." }), "\n", _jsx(Heading, { level: "2", id: "high-level-architecture", children: "High-Level Architecture" }), "\n", _jsx(Mermaid, { chart: "graph TD\n A[User Code] --> B[Session Object]\n B --> C{Memory Cache?}\n C -->|Hit| D[Return Data]\n C -->|Miss| E{SQLite Cache?}\n E -->|Hit| F[Load to Memory]\n E -->|Miss| G[CDN Manager]\n G --> H[jsDelivr CDN]\n H --> I[TracingInsights GitHub]\n I --> J[JSON Data]\n J --> K[Parse & Transform]\n K --> L[Save to SQLite]\n L --> F\n F --> D\n```python\n\n## Data loading pipeline\n\n### 1. Request Initiation\n\nWhen you access data, tif1 starts the loading pipeline.\n\n```python\nimport tif1\n\nsession = tif1.get_session(2025, \"Monaco Grand Prix\", \"Race\")\nlaps = session.laps # Triggers data loading\n```yaml\n\n### 2. Cache Lookup\n\ntif1 checks two cache tiers:\n\n**Tier 1: Memory LRU Cache**\n- Fastest (< 1ms)\n- Stores Python objects\n- Limited to 1024 items (configurable)\n- Cleared on process restart\n\n**Tier 2: SQLite Persistent Cache**\n- Fast (10-50ms)\n- Stores compressed JSON\n- Persists across restarts\n- Located in `~/.tif1/cache/`\n\n```python\n# Cache flow\nif key in memory_cache:\n return memory_cache[key]\nelif key in sqlite_cache:\n data = sqlite_cache[key]\n memory_cache[key] = data\n return data\nelse:\n # Fetch from CDN\n```yaml\n\n### 3. CDN Fetching\n\nIf data isn't cached, tif1 fetches from the CDN.\n\n**CDN Strategy:**\n- Primary: jsDelivr CDN (fast, global)\n- Fallback: Direct GitHub (slower, rate limited)\n- Never uses raw.githubusercontent.com (rate limits)\n\n```python\n# URL construction\nbase_url = \"https://cdn.jsdelivr.net/gh/TracingInsights/{year}@main\"\nlap_url = f\"{base_url}/laps/{gp}/{session}/driver_{driver}.json\"\n```python\n\n### 4. async parallel fetching\n\nFor multiple drivers, tif1 fetches in parallel using asyncio.\n\n```python\n# Sequential (slow)\nfor driver in drivers:\n data = fetch(driver) # 500ms each = 10s total for 20 drivers\n\n# Parallel (fast)\ntasks = [fetch(driver) for driver in drivers]\nresults = await asyncio.gather(*tasks) # 500ms total\n```python\n\n### 5. JSON Parsing\n\ntif1 uses orjson for fast JSON parsing.\n\n```python\nimport orjson\n\n# Parse JSON\ndata = orjson.loads(json_bytes)\n\n# Extract lap data\nlaps_data = data[\"laps\"]\n```yaml\n\n### 6. DataFrame Construction\n\nRaw JSON is converted to DataFrames.\n\n```python\n# Create DataFrame\ndf = pd.DataFrame(laps_data)\n\n# Rename columns (PascalCase)\ndf = df.rename(columns=COLUMN_RENAME_MAP)\n\n# Set data types\ndf[\"LapTime\"] = df[\"LapTime\"].astype(float)\ndf[\"Driver\"] = df[\"Driver\"].astype(\"category\")\n\n# Reorder columns\ndf = df[COLUMN_ORDER]\n```yaml\n\n### 7. Data Enrichment\n\ntif1 automatically enriches data.\n\n**Lap Data Enrichment:**\n- Add weather data to each lap\n- Calculate LapTimeSeconds helper\n- Add stint information\n- Categorize string columns\n\n**Telemetry Enrichment:**\n- Calculate acceleration (X, Y, Z)\n- Add DriverAhead information\n- Normalize distance (0.0 to 1.0)\n\n### 8. Cache Storage\n\nProcessed data is saved to both cache tiers.\n\n```python\n# Save to SQLite\nsqlite_cache.set(key, data)\n\n# Save to memory\nmemory_cache[key] = data\n```python ## Data types through the pipeline\n\n### Stage 1: Raw JSON\n```json\n{\n \"laps\": [\n {\n \"lap_number\": 1,\n \"lap_time\": 95.234,\n \"driver\": \"VER\",\n \"compound\": \"SOFT\"\n }\n ]\n}\n```yaml\n\n### Stage 2: Python dict\n```python\n{\n \"lap_number\": 1,\n \"lap_time\": 95.234,\n \"driver\": \"VER\",\n \"compound\": \"SOFT\"\n}\n```yaml\n\n### Stage 3: DataFrame\n```python\n LapNumber LapTime Driver Compound\n0 1 95.234 VER SOFT\n```yaml\n\n### Stage 4: enriched dataframe\n```python\n LapNumber LapTime Driver Compound AirTemp TrackTemp Stint\n0 1 95.234 VER SOFT 28.5 42.3 1\n```python\n\n## Performance Optimizations\n\n### HTTP/2 Multiplexing\n\ntif1 uses niquests for HTTP/2 support.\n\n**HTTP/1.1 (Old):**\n- One request per TCP connection\n- High overhead for multiple requests\n- Sequential processing\n\n**HTTP/2 (tif1):**\n- Multiple requests per connection\n- Reduced SSL/TLS overhead\n- Parallel processing\n\n### Lazy Loading\n\nData is only loaded when accessed.\n\n```python\nsession = tif1.get_session(2025, \"Monaco\", \"Race\")\n# No data loaded yet\n\nlaps = session.laps # Laps loaded now\n# Weather, telemetry still not loaded\n\nweather = session.weather # Weather loaded now\n```python\n\n### Categorical Optimization\n\nString columns are converted to categoricals for memory efficiency.\n\n```python\n# Before (high memory)\ndf[\"Driver\"] = [\"VER\", \"VER\", \"VER\", ...] # Each string stored\n\n# After (low memory)\ndf[\"Driver\"] = pd.Categorical([\"VER\", \"VER\", \"VER\", ...])\n# Strings stored once, integers used for values\n```python\n\n### Lib Selection\n\nChoose the right lib for your use case.\n\n**Pandas:**\n- Familiar API\n- Rich ecosystem\n- Good for small/medium data\n\n**Polars:**\n- 2x faster operations\n- 50% less memory\n- Better for large data\n\n## Error handling in the pipeline\n\n### Network Errors\n\n```python\ntry:\n laps = session.laps\nexcept tif1.NetworkError as e:\n # All CDN sources failed\n print(f\"Network error: {e}\")\n```python\n\n### Data validation errors\n\n```python\ntry:\n laps = session.laps\nexcept tif1.InvalidDataError as e:\n # JSON parsing or validation failed\n print(f\"Invalid data: {e}\")\n```text ### Cache Errors\n\n```python\ntry:\n cache = tif1.get_cache()\n cache.clear()\nexcept tif1.CacheError as e:\n # Cache operation failed\n print(f\"Cache error: {e}\")\n```python\n\n## Monitoring data flow\n\n### Enable debug logging\n\nSee detailed information about data flow.\n\n```python\nimport tif1\nimport logging\n\ntif1.setup_logging(logging.DEBUG)\n\n# Now you'll see:\n# - Cache hits/misses\n# - CDN requests\n# - Parsing times\n# - DataFrame construction\n```python\n\n### Check cache status\n\n```python\ncache = tif1.get_cache()\nprint(f\"Cache location: {cache.cache_dir}\")\n\n# Check if data is cached\n# (No direct API, but you can check load times)\nimport time\n\nstart = time.time()\nlaps = session.laps\nduration = time.time() - start\n\nif duration < 0.1:\n print(\"Cache hit (fast)\")\nelse:\n print(\"Cache miss (slow)\")\n```python\n\n### Monitor circuit breaker\n\n```python\ncb = tif1.get_circuit_breaker()\nprint(f\"State: {cb.state}\")\nprint(f\"Failures: {cb.failure_count}\")\n```python\n\n## Data flow patterns\n\n### Cold Start (First Load)\n\n```python\nUser Request\n ↓\nMemory Cache Miss\n ↓\nSQLite Cache Miss\n ↓\nCDN Fetch (2-5s)\n ↓\nJSON Parse (50-100ms)\n ↓\nDataFrame Build (100-200ms)\n ↓\nCache Save (50ms)\n ↓\nReturn Data\n```yaml\n\n**Total: ~2.5-5s**\n\n### Warm Start (Cached)\n\n```sql\nUser Request\n ↓\nMemory Cache Miss\n ↓\nSQLite Cache Hit\n ↓\nLoad from SQLite (10-50ms)\n ↓\nMemory Cache Save\n ↓\nReturn Data\n```python **Total: ~10-50ms**\n\n### Hot Start (In Memory)\n\n```python\nUser Request\n ↓\nMemory Cache Hit\n ↓\nReturn Data\n```python\n\n**Total: < 1ms**\n\n## Optimization Strategies\n\n### 1. Pre-warm Cache\n\n```python\ndef warm_cache(year):\n \"\"\"Pre-load all race data.\"\"\"\n for event in tif1.get_events(year):\n session = tif1.get_session(year, event, \"Race\")\n _ = session.laps # Trigger load\n```python\n\n### 2. Use Async Loading\n\n```python\nimport asyncio\n\nasync def load_fast():\n session = tif1.get_session(2025, \"Monaco\", \"Race\")\n laps = await session.laps_async() # 4-5x faster\n return laps\n```bash\n\n### 3. Batch Telemetry\n\n```python\n# Bad - Sequential\nfor driver in drivers:\n tel = driver.get_fastest_lap_tel()\n\n# Good - Parallel\ntels = session.get_fastest_laps_tels(by_driver=True)\n```sql\n\n### 4. select lib wisely\n\n```python\n# Small analysis - use pandas\nsession = tif1.get_session(2025, \"Monaco\", \"Race\", lib=\"pandas\")\n\n# Large analysis - use polars\nsession = tif1.get_session(2025, \"Monaco\", \"Race\", lib=\"polars\")\n```yaml\n\n## Summary\n\nUnderstanding tif1's data flow helps you:\n- Optimize performance with caching\n- Choose the right loading strategy\n- Debug issues effectively\n- Monitor system health\n- Make informed architecture decisions\n\nThe multi-tier caching, async fetching, and smart data transformations make tif1 4-5x faster than traditional approaches.\n\n---\n\n## Related Pages\n\n\n \n System architecture\n \n \n Data pipeline\n \n \n CDN management\n \n" })] }); } function MDXContent(props = {}) { const {wrapper: MDXLayout} = { ..._provideComponents(), ...props.components }; return MDXLayout ? _jsx(MDXLayout, { ...props, children: _jsx(_createMdxContent, { ...props }) }) : _createMdxContent(props); } return { default: MDXContent }; function _missingMdxReference(id, component) { throw new Error("Expected " + (component ? "component" : "object") + " `" + id + "` to be defined: you likely forgot to import, pass, or provide it."); }

Get Started

Guides

Concepts

Advanced

Reference

​High-Level Architecture

High-Level Architecture