How Your Phone Knows You're Walking, Running or Driving

Your phone knows when you are standing, walking, running, cycling or sitting in a car. It knows it in the background, without you opening an app, and it costs almost nothing in battery. It feels like magic; it is actually a very ordinary signal-processing pipeline stapled to a small classifier and a lot of defensive smoothing.

This post walks that pipeline end to end: what the sensors physically measure, why any one of them is insufficient, how raw samples become features, which models run on device, how fusion resolves ambiguity, and why the whole thing is designed around milliwatts rather than accuracy.

1. The problem, stated precisely

The field calls this Human Activity Recognition (HAR). Formally: given a multivariate time series from body-worn sensors, assign each short time window a label from a fixed set — typically still, walking, running, cycling, in_vehicle, on_foot, tilting.

Three constraints make it harder than a standard classification problem:

• Unknown placement. The phone can be in a front pocket, back pocket, a hand, a bag or a car mount. The same activity produces very different raw traces in each.
• Unknown orientation. The device frame is not the world frame, and it rotates continuously.
• A hard power ceiling. This must run 24/7 in the background on a budget of roughly a milliwatt, which rules out keeping GPS or even the gyroscope always on.

2. The accelerometer

A phone accelerometer is a MEMS device: a tiny silicon proof mass suspended on springs between capacitive plates. When the device accelerates, the mass lags, the gap between plates changes, and the capacitance change is read out as acceleration along one axis. Three of these, orthogonally arranged, give you a 3-vector in the device body frame.

The critical detail everyone gets wrong first: an accelerometer measures proper acceleration, not coordinate acceleration. A phone lying flat and perfectly still reads roughly (0, 0, +9.81) m/s², not zero. It is measuring the normal force from the table. A phone in free fall reads (0, 0, 0). Gravity is therefore always mixed into your signal, and it is usually the largest component.

// Android SensorEvent.values for TYPE_ACCELEROMETER, units m/s^2
flat on table   → [ 0.02, -0.05,  9.79 ]   // magnitude ~9.81, all gravity
held upright    → [ 0.01,  9.78,  0.31 ]   // gravity rotated into Y
free fall       → [ 0.00,  0.02,  0.01 ]   // ~0
walking, pocket → [ 1.8, 10.9, -2.4 ] ... oscillating at ~2 Hz

3. Separating gravity from motion

Before you can measure motion you must remove the 9.81 m/s² bias. Two standard approaches:

• Low-pass filter. Gravity is a near-DC component. A first-order IIR filter with a cutoff around 0.3–0.5 Hz estimates the gravity vector; subtract it to get linear acceleration. Cheap, but it lags during sustained turns.
• Orientation-based removal. Fuse gyroscope and accelerometer into a rotation quaternion (complementary or Kalman filter), rotate the measured vector into the world frame, and subtract (0, 0, g). More accurate, more expensive, and this is what TYPE_LINEAR_ACCELERATION and userAcceleration do for you.

const ALPHA = 0.8;            // ~0.5 Hz cutoff at 50 Hz sampling
let gravity = [0, 0, 0];

function onSample(ax, ay, az) {
  gravity = gravity.map((g, i) => ALPHA * g + (1 - ALPHA) * [ax, ay, az][i]);
  const linear = [ax - gravity[0], ay - gravity[1], az - gravity[2]];
  return linear;
}

4. The gyroscope

A MEMS gyroscope measures angular velocity in rad/s about each axis, using a vibrating mass and the Coriolis force: rotate a vibrating structure and it experiences a perpendicular deflection proportional to the rotation rate.

The gyroscope answers a question the accelerometer cannot: is the device rotating, and how smoothly? That distinction is what separates real locomotion from someone waving their phone around.

Gyroscopes also drift. Integrating angular rate to get an angle accumulates bias error of several degrees per minute, which is why gyro is always fused with the accelerometer's gravity reference (and sometimes the magnetometer) rather than used alone. And it is expensive: at 5 mW it can be 30× the accelerometer's draw, so most always-on pipelines keep the gyro off and only wake it when the accelerometer says something interesting is happening.

5. What each activity looks like

Locomotion is periodic, and that periodicity is the single most discriminative property in the whole problem. Human gait produces a step frequency band that is remarkably consistent across people.

Two rows in that table are the hard cases. Cycling vs driving overlap in speed and both have low body motion — the discriminator is pedal-cadence periodicity and lean dynamics from the gyroscope. Still-in-a-vehicle vs still-at-a-desk are nearly identical on the IMU when the car is stopped at a light; only GPS history and the vehicle's engine vibration signature (a high-frequency component around 20–50 Hz, requiring a higher sample rate to see) tell them apart.

6. GPS and the speed signal

GNSS gives you position, and the derivative of position is speed — a signal the IMU cannot produce at all, because integrating noisy acceleration twice diverges within seconds.

Speed is what makes in_vehicle tractable. But it comes with caveats:

• Power. A continuous fix is ~90 mW — three orders of magnitude above the accelerometer. It cannot be always-on.
• Latency. Cold start is 30+ seconds; indoors and in tunnels there is no fix at all.
• Noise floor. Horizontal error of 3–10 m means position-differenced speed is unusable below ~1 m/s. Modern receivers report Doppler-derived speed directly, which is far more accurate at low speeds — use location.speed, not your own delta.

The standard architecture is therefore IMU-triggered GPS: run the cheap accelerometer classifier continuously; when it emits a possible in_vehicle or a low-confidence result, request a short burst of location fixes to confirm, then shut the radio off again.

7. The recognition pipeline

Sampling

Human motion is band-limited: essentially all gait energy sits below 15–20 Hz. By Nyquist, 50 Hz sampling is comfortably sufficient, and 20 Hz is enough if you only care about walk/run/still. Higher rates buy you engine-vibration detection at the cost of power.

Crucially, samples are written into a hardware FIFO on the sensor hub — typically a few hundred to a few thousand entries — and the application processor is woken only once per batch. Batching is the single biggest power optimisation in the entire system, because the AP waking up costs far more than the sensor itself.

Conditioning

• Median filter (width 3) to kill single-sample spikes.
• Low-pass at 20 Hz to remove out-of-band noise.
• Split into gravity (< 0.3 Hz) and body acceleration (the rest).
• Optionally compute jerk — the time derivative of acceleration — which is a strong impact-detection feature.

8. Windowing and feature extraction

Classification happens on windows, never single samples. A single sample contains no information about periodicity. The near-universal choice, inherited from the UCI HAR dataset, is a 2.56-second window with 50% overlap — 128 samples at 50 Hz. That length covers 3–5 walking steps, enough to establish a frequency, while keeping end-to-end latency under three seconds.

// ---- time domain, per axis + magnitude ----
mean, std, median, min, max, range
mad                 // median absolute deviation, robust to spikes
energy = Σx²/N
iqr                 // interquartile range
entropy             // signal entropy of the value distribution
sma  = Σ(|x|+|y|+|z|)/N          // signal magnitude area, activity intensity
correlation(x,y), correlation(x,z), correlation(y,z)
zero_crossing_rate
autocorr_peak_lag   // ← step period; the highest-value single feature

// ---- frequency domain, via 128-point FFT ----
dominant_frequency          // argmax of spectrum, excluding DC
dominant_magnitude
spectral_centroid           // "brightness" of the motion
spectral_entropy            // periodic (low) vs chaotic (high) → walk vs shake
band_energy[0.5-3, 3-8, 8-15, 15-25 Hz]
harmonic_ratio              // energy at 2f₀ / energy at f₀

9. Models: classical vs deep

The honest summary: hand-crafted features plus gradient-boosted trees remain extremely competitive, and they quantise to a few kilobytes that fit in a sensor hub. 1-D CNNs win when placement and population diversity are wide, because they learn placement-robust filters that no one would hand-design. Accuracy differences of two or three points matter much less than the smoothing layer described next.

# input: (batch, 128 timesteps, 6 channels: ax ay az gx gy gz)
model = Sequential([
    Conv1D(64, 5, activation='relu', input_shape=(128, 6)),
    BatchNormalization(),
    Conv1D(64, 5, activation='relu'),
    MaxPooling1D(2),
    Conv1D(128, 3, activation='relu'),
    GlobalAveragePooling1D(),      # placement/phase invariance
    Dropout(0.4),
    Dense(6, activation='softmax') # still/walk/run/cycle/vehicle/unknown
])
# → TFLite, int8 post-training quantisation → ~120 KB, ~2 ms/window on a mid-range SoC

10. Sensor fusion

"Sensor fusion" is used loosely for two different things, and it is worth separating them:

• Low-level (state) fusion. A complementary or Kalman filter combines gyro (accurate short-term, drifts) with accelerometer and magnetometer (noisy short-term, stable long-term) to maintain an orientation estimate. This is what lets you rotate the acceleration vector into world coordinates.
• High-level (decision) fusion. Features or class probabilities from multiple sources are combined — concatenated before the classifier (early fusion), or combined as votes/probabilities afterwards (late fusion). Early fusion usually wins when all sensors are available; late fusion degrades gracefully when one is missing, which matters because GPS regularly disappears.

Non-inertial context signals matter more than people expect. A connected car Bluetooth profile is nearly conclusive for in_vehicle. A barometer showing steady pressure change distinguishes stairs from a lift. Wi-Fi scan stability implies you have not left the building. These cost almost nothing and are already being collected.

11. Smoothing, hysteresis and state machines

A raw per-window classifier flickers. At 2 windows per second, even a 95%-accurate model produces a wrong label roughly every ten seconds — and a UI that says "driving" for one second while you walk feels broken even though the model is fine.

The fix is temporal modelling on top of the classifier:

• Hidden Markov Model. Treat classifier outputs as emissions and encode a transition matrix with strong self-transition probabilities. Viterbi decoding over a rolling buffer removes isolated flips almost entirely. This is the single highest-leverage post-processing step.
• Physically impossible transitions. Still → in_vehicle at 70 km/h in one window cannot happen. Zero out those transitions.
• Hysteresis / dwell time. Require N consecutive windows above a threshold to enter a state, and a higher N to leave it. Asymmetric thresholds prevent oscillation at the boundary.
• Confidence gating. When max probability < 0.6, emit unknown and hold the previous state rather than guessing.

const HISTORY = 7;                 // ~3.5 s at 2 windows/s
const ENTER = 5, EXIT = 3;         // asymmetric dwell counts
const buf = [];
let state = 'unknown';

function update(pred, confidence) {
  if (confidence < 0.6) return state;          // gate low-confidence windows
  buf.push(pred);
  if (buf.length > HISTORY) buf.shift();

  const counts = buf.reduce((m, p) => (m[p] = (m[p] || 0) + 1, m), {});
  const [top, n] = Object.entries(counts).sort((a, b) => b[1] - a[1])[0];

  const need = top === state ? EXIT : ENTER;   // harder to enter than to stay
  if (n >= need) state = top;
  return state;
}

12. The power budget

Every architectural decision above is downstream of this chart. The reason the pipeline lives on a sensor hub, batches through a FIFO, keeps the gyro off and treats GPS as a last-resort tiebreaker is that the cheap path is roughly 600× cheaper than the expensive one.

• Tiered activation. Accelerometer always on → if variance crosses a threshold, wake the gyro → if the class is ambiguous or vehicle-like, request GPS.
• Batching. Deliver 200 samples once rather than waking the AP 200 times.
• Duty cycling. When the state has been still for minutes, drop to a 5-second sample every 30 seconds. A stationary phone needs almost no attention.
• Hardware significant-motion sensor. Both platforms expose a one-shot wake-up trigger implemented in silicon; use it as the entry point instead of polling.

13. The platform APIs

You almost never build this yourself in a product. Both platforms ship a calibrated, battery-optimised classifier that has been trained on far more labelled data than you can collect.

val transitions = listOf(
  ActivityTransition.Builder()
    .setActivityType(DetectedActivityType.WALKING)
    .setActivityTransition(ActivityTransition.ACTIVITY_TRANSITION_ENTER)
    .build(),
  ActivityTransition.Builder()
    .setActivityType(DetectedActivityType.IN_VEHICLE)
    .setActivityTransition(ActivityTransition.ACTIVITY_TRANSITION_ENTER)
    .build()
)

ActivityRecognition.getClient(context)
  .requestActivityTransitionUpdates(
      ActivityTransitionRequest(transitions), pendingIntent)

14. Where it still fails

• Stationary vehicle. Stopped at a red light, the IMU sees a desk. Systems rely on state persistence and GPS history rather than the current window.
• Train and bus passengers. Smooth motion, no body movement, high speed — frequently labelled in_vehicle, which is technically right but useless if you wanted to detect driving specifically.
• Walking inside a moving vehicle. Two superimposed motion sources; the classifier sees a mixture that matches no training example.
• Escalators and lifts. Stationary body, changing altitude — resolved by the barometer, not the IMU.
• Population shift. Models trained on healthy young adults degrade sharply on elderly gait, mobility aids, or children, whose step frequency and amplitude sit outside the training distribution.
• Placement shift. Handbag-carried phones are the classic failure mode: the bag's own pendulum swing adds a second periodic component near the step frequency.

15. Build it yourself

A working prototype is a weekend of effort. The path that actually converges:

• Collect. Log accel + gyro at 50 Hz with a label you press before each session. Get at least 20 minutes per class, across pocket / hand / bag placements, from more than one person. Placement diversity matters more than volume.
• Window. 2.56 s, 50% overlap. Split train/test by subject, never randomly by window — overlapping windows leak across the split and inflate accuracy by 10+ points. This is the mistake in most HAR tutorials.
• Baseline. Compute std(‖a‖) and dominant_frequency, fit a decision tree, look at the confusion matrix. That matrix tells you which pair to work on next.
• Improve. Add the full feature set and gradient boosting, or switch to a 1-D CNN on raw windows. Compare against the baseline honestly.
• Smooth. Add rolling mode plus hysteresis. Your perceived accuracy will jump more from this than from any model change.
• Ship. Quantise to int8, export TFLite / Core ML, and measure battery over a full day before you believe any of it.

Public datasets to start from: UCI HAR (30 subjects, 6 activities, the standard baseline), WISDM, PAMAP2 (multi-sensor, includes heart rate), and SHL — the Sussex-Huawei Locomotion set, which is the realistic one for transport modes because it includes bus, train, subway and car.