Digital Video — From camera frames to efficient transmission

Digital video is, at heart, a rapid sequence of still images. Yet raw video is enormous — far too large to store or transmit as-is — so almost everything interesting about video technology is about representing moving pictures efficiently while keeping them good enough for the job. This article walks from a single frame to the compressed streams that make live and stored video practical.

It covers generic, industry-standard concepts. How any particular system is built — including specific implementation choices — is a separate matter and not the subject here.

Frames: video is pictures in sequence

A video is a series of frames shown fast enough that the eye perceives smooth motion. Two basic properties describe the raw image stream:

• Resolution — the number of pixels per frame (e.g. 1920×1080). More pixels means more detail and more data.
• Frame rate — how many frames per second (fps). Higher rates give smoother motion and more data.

Multiply resolution × colour depth × frame rate and the raw data rate becomes very large very quickly — which is exactly why compression exists.

Colour representation

Each pixel’s colour must be encoded. Two ideas matter in practice:

• Colour models — images are often stored not as red/green/blue but as luma (brightness) plus chroma (colour) components, because the eye is far more sensitive to brightness detail than to colour detail.
• Chroma subsampling — exploiting that fact, systems store colour at lower resolution than brightness (e.g. “4:2:0”), cutting data with little visible loss. It is one of the oldest and most effective tricks in video.

Why compression is essential

Uncompressed video is impractical to store or send at scale, so it is compressed by a codec (coder–decoder). Compression can be:

• Lossless — perfectly reversible, but only modest size reduction.
• Lossy — discards information the eye is least likely to miss, achieving very large reductions. Almost all delivered video is lossy; the art is discarding what matters least.

Intra- and inter-frame coding

The biggest savings come from the fact that consecutive frames are usually very similar. Codecs exploit two kinds of redundancy:

• Intra-frame (spatial) — compress a single frame on its own, like a still image, removing redundancy within the picture. Such frames (often called keyframes or I-frames) can be decoded independently.
• Inter-frame (temporal) — describe a frame in terms of what changed from earlier (and sometimes later) frames — motion and differences — rather than storing it whole. These predicted frames are much smaller but depend on other frames to decode.

A stream is therefore a mix of independent keyframes and cheaper predicted frames. Keyframe frequency is a key trade-off: frequent keyframes make seeking and error-recovery easier but cost more data; sparse keyframes compress better but are more fragile.

Bitrate: the central dial

Bitrate — data per second — is the master control over quality and size. For a given codec and content, more bitrate means better quality and larger files/streams. Encoding can be:

• Constant bitrate (CBR) — steady data rate, predictable for transmission.
• Variable bitrate (VBR) — spend more bits on complex scenes, fewer on simple ones, for better quality at a given average size.

Complex, fast-moving or noisy content needs more bits for the same quality than simple, static content — content itself drives the cost.

Latency, live versus stored, and buffering

Whether video is stored (play later) or live (play now) changes the priorities:

• Stored/on-demand — quality and compression efficiency dominate; a few seconds of start-up buffering is acceptable, and the encoder can look ahead across the whole clip.
• Live/real-time — latency (delay between capture and display) becomes critical, which limits how much look-ahead and buffering can be used. Interactive uses (two-way communication, remote control) demand especially low latency, forcing trade-offs against compression efficiency and error resilience.

Buffering — holding a little data ahead of playback — smooths out network variation so playback does not stutter, at the cost of added delay. Live systems keep buffers small; on-demand systems can afford more.

Networks: packet loss and adaptation

Sending video over real networks introduces problems compression alone does not solve:

• Packet loss — because predicted frames depend on earlier ones, a lost packet can affect more than one frame until the next keyframe; error-resilience techniques and well-placed keyframes limit the damage.
• Bandwidth variation — available throughput changes over time. Adaptive delivery switches between different quality/bitrate versions as conditions change, trading resolution for continuity when the network tightens.
• Jitter — uneven arrival timing, absorbed by buffering.

Quality trade-offs

There is no single “best” setting — only the right balance for the use:

• resolution and frame rate versus bitrate;
• compression efficiency versus latency and error-resilience;
• keyframe frequency versus seek/recovery ability;
• perceived quality versus file/stream size.

Measuring “quality” is itself subtle — simple pixel-difference metrics do not match human perception well, so perceptual metrics and human judgement are often used.

Representative applications

• Streaming and media — on-demand and live delivery to many viewers.
• Video communication — calls and conferencing, where low latency is paramount.
• Surveillance and monitoring — continuous capture, storage and review.
• Machine vision pipelines — video as the input to detection and measurement (see Computer Vision).

These are examples, not a fixed list — anywhere moving images must be stored or moved efficiently, these same concepts apply.

Combining with other technologies

• With computer vision — the same frames that are compressed for viewing can be analysed for content; running analysis where the video is produced avoids shipping raw streams (see Computer Vision).
• With edge computing — encoding, and often analysis, near the camera cuts the data that must travel and the latency of acting on it (see Edge Computing).
• With connected systems — video is often the heaviest traffic on a link, so how it is compressed and delivered shapes the whole communication design.

Limitations and cautions

• Raw video is huge — compression is not optional at any realistic scale.
• Latency and compression pull against each other — you cannot maximise both; interactive video accepts lower efficiency for lower delay.
• Lossy means lost — heavily compressed video discards real detail; how much is acceptable depends entirely on the purpose (viewing versus measurement can differ).
• Networks are imperfect — packet loss and variable bandwidth must be designed for, not assumed away.

Understood as a chain — capture, colour, compression, and delivery over an imperfect network — digital video is a continual negotiation between quality, size and delay, tuned to whether a human is watching a film or a machine is reacting in real time.

Sources & further reading

• Richardson, I. E., The H.264 Advanced Video Compression Standard, Wiley — a clear, thorough treatment of modern video coding.
• Wiegand, T., Sullivan, G. J., Bjøntegaard, G. & Luthra, A., “Overview of the H.264/AVC Video Coding Standard,” IEEE Transactions on Circuits and Systems for Video Technology, 2003 — the standard’s own overview.
• Poynton, C., Digital Video and HD: Algorithms and Interfaces, Morgan Kaufmann — a reference on resolution, colour and signal representation.
• ITU-T Recommendations H.264 (AVC) and H.265 (HEVC) — the underlying international coding standards.