After you’ve set up a Mixpanel Data Pipeline and confirmed your first export is landing in your warehouse, the next question is always the same: how do I actually query this data correctly?
This is where most teams hit friction. The pipeline is running, files are arriving in BigQuery or Snowflake, but the event counts don’t match Mixpanel’s UI, user identity looks wrong, and nobody’s sure how to handle the JSON properties structure. None of that is a bug — it’s just how the pipeline works, and understanding the data model upfront saves hours of debugging later.
This post covers what JSON Pipelines export, how the data is structured, what the incremental pipeline system means for your queries, and the specific things you need to handle yourself — like GDPR deletions and identity resolution — that the pipeline doesn’t do automatically.
What JSON Pipelines Export and Why Everything Is Under properties
JSON Pipelines support three data sources: events, user profiles, and identity mappings. You can create pipelines for any combination of these, and they each write to different tables or directories in your destination.
The design decision that surprises most people first is that all event properties and user profile properties are stored under a single properties JSON key rather than as individual columns. Every event row has a fixed set of top-level fields, and then everything else — every custom property, every default Mixpanel property, every tracking attribute your implementation sends — lives inside a JSON object in the properties column.
For events, the top-level fields in BigQuery look like this:
device_id— the device identifier used while a user is anonymousdistinct_id— the unique identifier for the user who triggered the eventevent_name— the name of the eventinsert_id— the deduplication key Mixpanel uses to prevent counting the same event twiceproperties— a JSON object containing every property on the eventtime— a timestamp of when the event occurreduser_id— the identifier used to track a user across devices after identification
For user profiles, you get distinct_id and properties. For identity mappings, you get distinct_id and resolved_distinct_id.
The column names are consistent across warehouses, but the column types differ. BigQuery uses JSON for the properties column. Snowflake uses VARIANT. Redshift and Databricks have their own corresponding types. The querying syntax for extracting values from those JSON columns is different in each warehouse — know your warehouse’s JSON extraction functions before you start building models on this data.
Understanding the Three Data Sources
Events
Event data is the core export. Each row represents a single event as it was ingested by Mixpanel. This is important to understand: the pipeline exports events as they appear at ingestion time, not as they appear in the Mixpanel UI after identity resolution has occurred.
This means if a user triggered events before being identified, those early events carry the anonymous device_id or pre-identification distinct_id, not the resolved identifier you’d see for that user in Mixpanel reports. More on how to handle this below.
User Profiles
User profiles export to a single table or directory called mp_people_data. Unlike events — which are additive and append new rows — the people data export replaces the entire table on each run with the current state of all user profiles.
This means user profiles are always current as of the last export, but you don’t get a history of how profiles changed over time. If you need to track profile attribute changes historically, you’ll need to snapshot the people table yourself at regular intervals before each replacement.
Identity Mappings
The identity mappings table is the bridge between anonymous and identified user behavior. It contains two fields: distinct_id (the original identifier) and resolved_distinct_id (what Mixpanel resolved that identifier to after applying its identity merge logic).
This table is only relevant if your Mixpanel project has ID merge enabled. If you’re using the simplified ID merge API or the original API, you’ll need this table to accurately count unique users in your warehouse. Without it, your warehouse user counts will differ from what Mixpanel shows in its UI.
The Incremental Pipeline System: What Changed and Why It Matters
As of September 2025, all JSON pipelines have been migrated to an incremental export system. If you’re setting up a new pipeline or working with an existing one, understanding this system changes how you think about file structure, timing, and data freshness.
The old system used a daily sync process that would go back and re-export days where data had changed — catching late-arriving events, GDPR deletions, and other retroactive changes. This meant files for a given day could be replaced or updated on subsequent sync runs.
The new incremental system works differently. Instead of syncing backward, it continuously exports new data as it arrives. Late-arriving events are exported when they come in, regardless of how late they are. There’s no longer a 10-day sync window restriction on late data.
What this changes in practice:
More files, smaller files. Under the old sync system, a day’s data might get consolidated into a smaller number of files when the day was re-synced. With incremental exports, each pipeline run appends new files for new data rather than replacing existing files. Expect more files per day in your storage destination. If you’re building pipelines that process files, design them to handle multiple files per day partition rather than assuming one file per day.
Export timing is no longer fixed. Hourly pipelines no longer run on a strict schedule. Each run starts after the previous one completes rather than at a fixed time offset. The “hourly” label describes the export frequency intent, not a guaranteed wall-clock schedule. Build any downstream processes with this variability in mind.
Backfill completes before regular processing starts. When you create a new pipeline with a historical backfill, the full backfill exports first before any incremental processing begins. If you create a pipeline on January 15th with a backfill from January 1st, all events that arrived in Mixpanel before January 15th export as the backfill, then incremental processing picks up new events arriving after that point. Don’t expect incremental exports to start until the backfill finishes.
Past pipeline logs are gone after migration. Existing pipelines that migrated to the incremental system had their logs reset. Historical job log lines are no longer visible — only logs from the new incremental jobs forward.
GDPR deletions no longer propagate through sync. Under the old system, sync could occasionally remove deleted data from your warehouse when a day was re-synced. This was never guaranteed behavior, but it happened. With incremental pipelines, it no longer happens at all. You are fully responsible for deleting GDPR-related data from your warehouse when a deletion occurs in Mixpanel. More on this below.
User Identity Resolution: The Part Most Teams Get Wrong
This is the area where warehouse queries diverge most significantly from what Mixpanel shows in its UI, and it’s worth spending time understanding properly.
Mixpanel’s UI shows resolved user counts — it applies identity merge logic to connect anonymous events with identified events and presents a unified user journey. Pipeline exports don’t do this. They export events as ingested, with whatever distinct_id was sent at the time.
This means a single real user might appear in your events table with two or more distinct IDs — an anonymous ID from before they logged in, and their actual user ID after identification. If you count distinct_id values in your warehouse to get unique user counts, you’ll overcount relative to Mixpanel.
The identity mappings table fixes this. It maps original identifiers to their resolved equivalents. The rule for querying correctly is:
Use resolved_distinct_id from the identity mappings table wherever it’s available. Where there’s no mapping — meaning that identifier was never merged with another — fall back to the distinct_id from your events or people table.
Mixpanel has specific SQL examples for doing this join correctly in BigQuery and Snowflake. Use those examples as your starting pattern. Building this join into your data models from day one is much easier than retrofitting it after analysts have already built dashboards that show wrong user counts.
GDPR Deletions: Your Responsibility in the Warehouse
This is the limitation that has the biggest compliance implications and the one most teams don’t discover until they’re in an audit.
When a user is deleted from Mixpanel via the GDPR deletion API, that deletion is processed in Mixpanel’s own storage. It does not automatically propagate to data that has already been exported to your warehouse via pipelines.
With the old sync-based system, there was an occasional side effect where re-syncing a day could remove deleted data from the warehouse — but this was never guaranteed and was not the intended behavior. The incremental pipeline system removes even that unreliable side effect.
The result is that your warehouse is not automatically GDPR compliant just because Mixpanel processed the deletion. You need a separate process that:
- Receives notification that a GDPR deletion has been processed in Mixpanel (via the deletion API or your own request tracking)
- Identifies all records in your warehouse tables associated with the deleted user — both in events tables and in the people table
- Deletes or anonymizes those records according to your compliance requirements
Build this process before you need it, not after a data subject access request surfaces it. The pipeline will keep adding new data for all users, including data for users who were previously deleted in Mixpanel if they return and create a new identity. Make sure your deletion process handles the full scope of data across all warehouse tables that receive pipeline exports.
Data Integrity: Preventing Duplicate Exports
The pipeline enforces one constraint that’s worth understanding if you’re ever tempted to create multiple pipelines writing to the same destination: you cannot have multiple event pipelines exporting to the same destination with overlapping date ranges.
If you have a pipeline exporting to BigQuery dataset my_dataset covering January 1–31, you can’t create another pipeline for the same dataset covering January 15 through February 15. The overlap on January 15–31 would cause duplicate data.
This constraint is enforced at pipeline creation. If you need to restructure your pipelines, delete the existing one and create new pipelines with non-overlapping date ranges.
Common Questions That Come Up After Setup
Why doesn’t my hourly pipeline run exactly every hour?
Incremental pipelines start each run after the previous one completes, not on a fixed wall-clock schedule. A run that takes 45 minutes means the next run starts 45 minutes after the previous one began, not at the next hour mark. This is by design — it ensures complete export without cutting off data mid-stream.
Can I export only specific events or filter which properties come through?
Not with JSON pipelines created through the UI. All events and all properties are exported as ingested. The older API-based pipelines (Raw and Schematized) do support event whitelisting and property filtering, but those are a different pipeline type with a different setup process.
Can I customize the folder structure or prefix in my storage bucket?
No. The export paths are fixed by Mixpanel and follow the structure <BUCKET_NAME>/<PROJECT_ID>/mp_master_event/<YEAR>/<MONTH>/<DAY>/ for events. Custom subfolders or prefixes aren’t currently supported.
Why does the event count in the pipeline UI look different than before?
With incremental pipelines, the event count shown per task represents the total events processed per batch rather than events per calendar day. Because batches can span multiple days, the number looks different from the old per-day counts. This is a display change, not a data quality issue.
Why does People data replace the table on each export rather than appending?
User profiles are mutable — they change over time as user attributes update. Appending profile exports would give you multiple rows per user with different property values and no clear way to know which is current. Full replacement ensures you always have one current row per user. If you need profile history, snapshot the table before each replacement.
Building on Top of Pipeline Data: What to Do First
If you’re setting up warehouse queries on top of a new JSON pipeline, here’s the sequence that prevents the most common mistakes:
Build the identity resolution join first. Before any analyst touches the event data, create a model or view that joins events to the identity mappings table and resolves distinct_id to resolved_distinct_id. Make this the canonical events table that everyone queries, not the raw export.
Handle timezone conversion at the model layer. All pipeline data exports in UTC. Your project’s timezone is almost certainly not UTC. Build the timezone conversion into your base models so every downstream query uses the correct local time without analysts having to remember to convert.
Account for multiple files per day in your ingestion logic. If you’re building a process that reads new files from storage and loads them into warehouse tables, design it to handle multiple files arriving for the same day partition across multiple pipeline runs.
Set up GDPR deletion handling before you need it. Document which warehouse tables contain personal data from pipeline exports, and build the deletion process now. It’s significantly easier to implement proactively than retroactively.
Use the warehouse-specific SQL examples from Mixpanel’s docs for event counting. The partitioning and storage structure differs between BigQuery, Snowflake, and other warehouses. Generic COUNT queries can produce inaccurate results. Start from the official examples and adapt from there.
The Bottom Line
JSON Pipelines are well-designed for getting Mixpanel data into your warehouse reliably and continuously. The incremental system makes late data handling significantly more robust than the old sync approach. But the pipeline handles transport — the data modeling, identity resolution, GDPR compliance, and timezone handling are your responsibility on the warehouse side.
The teams that get the most value out of pipeline exports are the ones who invest in a clean base data model before opening up warehouse access broadly. Get the identity resolution join right, handle UTC conversion at the foundation, and design for GDPR deletion from the start. Everything downstream from that foundation becomes significantly more reliable.