Compressing Fleet Sensor Data Without Losing Fidelity
Sep 25, 2026 Resolute Dynamics
A fleet’s sensors produce a flood of data, and storing all of it at full fidelity gets expensive fast. Deleting freely is no answer either, because the moments that matter most, a hard brake or a fault, are exactly the ones you cannot afford to lose.
The real answer is not a single compression setting for everything. It is matching the fidelity of each data stream to how much that stream is worth. This guide covers the two families of compression, the techniques that shrink fleet data, and how to keep what matters while cutting the rest.
The Fidelity vs Cost Trade-Off
The trade-off is simple to state: higher fidelity costs more to store, lower fidelity risks losing something you needed. Every compression choice sits somewhere between those two poles, and the goal is to keep the important detail while shrinking everything else.
Why One Setting for the Whole Fleet Is Wrong
Applying one compression level to all data is a mistake, because fleet data is not all equally valuable. The seconds around a collision and a truck idling in a yard are worth very different things, yet a single setting treats them the same. It either keeps the idle data in wasteful detail or crushes the crash data into uselessness. Matching the method to the data avoids both.
Lossless vs Lossy Compression
There are two families of compression: lossless, which keeps every point, and lossy, which drops data on purpose. Choosing between them is the first decision for any data stream.
Lossless Compression
Lossless compression keeps every data point and simply packs it more efficiently, like zipping a file. The data is fully recoverable later, and time-series stores can reach around 90% storage reduction without dropping a single point. A common technique is delta encoding, which stores the small difference between one reading and the next rather than the full value each time.
Lossy Compression
Lossy compression throws away data on purpose, assuming the tiny wiggles add no real value. Also called downsampling, it removes the small, rapid fluctuations that are usually just noise. It saves far more space than lossless, but the discarded detail is gone for good, so it fits data where the fine grain does not matter.
The Techniques That Shrink Fleet Data
A few techniques do most of the work, each with its own reduction and its own trade-off. The table below sets them side by side.
| Technique | Typical reduction | Keeps | Watch out for |
|---|---|---|---|
| Lossless (with delta encoding) | Up to ~90% | Every point | Least space saved |
| Dead-band filtering | ~80–90% | Big changes | Flattens slow drifts |
| Swinging door trending | 95–98% fewer points | Trend shape | Needs tuning |
Dead-Band Filtering
Dead-band filtering sends a new value only when it changes beyond a set margin. A temperature holding near 100 degrees reports nothing until it moves past the threshold, which typically cuts data by 80 to 90% and is excellent for noise and alarms. Its weakness is slow drift: a gradual change below the margin shows up as a flat staircase, which can hide a trend that was quietly building.
Swinging Door Trending
Swinging door trending keeps only the points that change the shape of a trend. It holds an error band that swings like a door and drops any reading that fits the current direction, achieving 95 to 98% fewer points while the reconstructed trend stays nearly indistinguishable from the raw data. Unlike dead-band filtering, it preserves slopes and gradual ramps, so it captures a real drift instead of flattening it.
Downsampling and Delta Encoding
Downsampling and delta encoding cover the simpler end of the range. Downsampling keeps one reading out of many on a fixed rule, which suits data that changes slowly. Delta encoding, which is lossless, stores the difference between consecutive readings, so a signal that barely changes compresses to almost nothing without losing a point.
Match the Method to the Data
The core rule is to keep safety and compliance data lossless, and reduce routine telemetry hard. Sorting each stream into one of these two tiers is what balances fidelity against cost.
Data That Must Stay Lossless
Some data has to be kept at full fidelity, every point intact. The window of data around a collision, harsh-braking and speed-limiter events, fault codes, and any record needed for compliance all fall here. These are the moments a fleet may have to reconstruct exactly, in an investigation or an audit, so a lossy method that smooths them over is not an option.
Data That Can Be Reduced
Most data is routine and can be shrunk hard without losing anything useful. Steady cruising speed, a stable engine temperature, and position on an empty stretch of highway carry little new information minute to minute. Dead-band filtering or swinging door trending reduces these streams dramatically, which is where the bulk of the storage saving comes from.
Where to Compress: Edge vs Server
Compression can happen on the vehicle or on the server, and the strongest setups use both. Each place offers something the other cannot.
Compressing at the edge, on the vehicle, runs in real time and cuts both network and storage cost as the data is created, since redundant readings never get sent. Compressing on the server runs on the full dataset with the complete picture, which allows tighter compression but saves nothing on the network.
The best practice combines them: downsample at the point of collection, then layer lossless compression on the stored data. A vehicle data capture platform that holds the stored data is where that second, lossless layer does its work.
Getting It Right
Getting compression right comes down to tiering the data first, then tuning each tier. Deciding what matters before choosing a ratio keeps a fleet from crushing the wrong stream.
- Sort each data stream into must-keep-lossless or safe-to-reduce.
- Keep event, safety, and compliance data lossless, with full fidelity around every event.
- Apply dead-band or swinging door to routine telemetry, tuned to keep real trends.
- Compress at the edge and again on the server for the best of both.
Frequently Asked Questions
What is the difference between lossless and lossy sensor data compression?
Lossless compression keeps every data point and packs it tighter, while lossy compression discards data on purpose. Lossless data is fully recoverable; lossy data trades lost detail for much smaller size. Fleets use lossless for critical data and lossy for routine telemetry.
How much can fleet sensor data be compressed?
It depends on the method: lossless can reach around 90% smaller, while lossy methods can drop 95 to 98% of the points. Routine telemetry compresses the most because it changes little. Critical data is kept lossless, so it saves less but loses nothing.
Which fleet data should never be compressed with a lossy method?
Never use lossy compression on event windows, crash data, safety and speed-limiter events, fault codes, or compliance records. These may have to be reconstructed exactly in an investigation or audit. A lossy method that smooths them over destroys the detail that matters most.
What is dead-band filtering?
Dead-band filtering only sends a new value when it changes beyond a set margin. It typically cuts data by 80 to 90% and is great for noise and alarms. Its downside is that very slow changes can appear as flat steps rather than a gentle slope.
Should compression happen on the vehicle or in the cloud?
Ideally both. Compressing on the vehicle cuts network and storage cost in real time, while compressing on the server allows tighter results on the full dataset. The best practice is to downsample at the edge, then apply lossless compression to the stored data.