Computer Vision — How machines make sense of images

A camera turns light into an image — a grid of pixels. Computer vision is the step after that: extracting information from those pixels that a system can act on. Where a person looks at a photo and immediately sees “a ripe tomato, front-left, partly hidden by a leaf”, computer vision tries to produce a similar structured answer automatically — what is in the image, where it is, and sometimes how much or how many — by detecting patterns in the pixels, not by understanding the scene the way a person does.

It is a genuinely useful tool, and also an easy one to overestimate. A model that works beautifully on the images it was built for can fail on images that look, to us, almost identical. Understanding both sides is what makes it dependable.

What computer vision is

It helps to separate two things that often get blurred together:

Visible imaging is capturing an image in the light the human eye uses — an ordinary camera producing a colour picture. (This is the counterpart to thermal imaging, which captures infrared instead; see Thermal Imaging.)
Computer vision is the computational step that extracts meaning from that image: detecting, locating, classifying, measuring or tracking things in it.

A camera gives you pixels. Computer vision gives you a description of what those pixels represent.

How it works

At the lowest level an image is just numbers — for a colour image, three intensity values (red, green, blue) per pixel. Everything computer vision does is built on finding structure in those numbers.

Broadly, there are two families of technique, and modern systems mix them:

Classical image processing works with explicit rules: thresholds, edges, colour ranges, shapes and hand-designed features. It is fast, predictable and easy to reason about — well suited to controlled conditions (consistent lighting, known backgrounds).
Learned models (today usually deep neural networks) are trained on many labelled examples and learn their own features. They handle far messier, more varied images than hand-written rules can — at the cost of needing representative training data and being harder to interpret.

Whichever approach, the common tasks are:

Classification — what is this image (or region) of?
Detection — find each object and where it is (a bounding box).
Segmentation — label which pixels belong to which object.
Tracking — follow an object across frames of video.
Measurement — estimate size, count, position or condition — which usually requires calibration to convert pixels into real-world units.

What it enables

In practical terms, computer vision lets a system notice and act on what is in front of a camera without a person watching: count items, spot a defect, read a gauge, check that something is present or absent, locate a part for a robot, or flag a condition worth attention. It turns a camera from something that records into something that reports.

Why image quality and context matter

The image itself is one of the biggest factors in whether computer vision works — alongside model quality, training data, calibration and overall system design. A great deal of real-world difficulty lives in the image:

Lighting changes everything — shadows, glare, colour casts and low light can make the same object look completely different.
Focus, resolution and motion blur set a hard limit on what can be seen. Small objects are especially demanding: detecting a tiny fruit needs enough pixels on that fruit to work with.
Occlusion, perspective and scale — objects overlap, hide behind each other, and change apparent size with distance.
Background clutter makes the thing of interest harder to separate from everything else.
Domain shift — a learned model performs well on data resembling what it was trained on and can degrade sharply outside it (a new camera, season, or site). This is one of the most common reasons a system that “worked in the demo” struggles in the field.

None of these are reasons not to use computer vision. They are the reasons it is an engineering problem — the camera, optics, lighting and data matter as much as the algorithm.

Representative applications

The same core ability — locating and identifying things in an image — supports very different uses:

Industrial inspection: detecting defects, verifying assembly, reading labels or gauges.
Counting and sorting: counting items on a line, grading produce, sorting by size or class.
Guidance and robotics: locating parts or targets so a machine can act on them.
Agriculture and environment: detecting plants, fruit or features and assessing condition across a scene.
Safety and monitoring: noticing people, vehicles or events in a camera feed (with due care for privacy).

These are examples of where it fits, not a fixed list — anywhere useful information is visible in an image is a candidate.

Combining with other technologies

Computer vision is at its most useful as one part of a larger system:

With thermal imaging: the visible camera locates and identifies an object; the thermal camera reports its surface temperature. Aligning the two lets a temperature reading be attached to a specific, recognised object rather than an anonymous region (see Thermal Imaging).
With edge computing: running the vision locally, near the camera, gives real-time results without shipping raw video elsewhere — better for latency, bandwidth and privacy.
With data and analytics: a stream of detections becomes counts, trends and alerts, so the system surfaces what matters instead of producing more footage to review.

Limitations and cautions

It recognises patterns; it does not understand a scene. Confident output can still be wrong, especially on inputs unlike the training data.
Learned models can carry bias from their data and produce false positives and negatives; the cost of each error should be considered deliberately.
Measurement needs calibration — turning pixels into real-world sizes or positions requires knowing the camera geometry.
Privacy matters: cameras can capture people and identifying detail. Data minimisation, on-device processing and clear purpose limits are part of doing this responsibly.

Used with these in mind — good imaging, representative data, honest evaluation — computer vision is a dependable way to turn what a camera sees into something a system can use.

Example explored by Mansonix

Mansonix is exploring a horticultural application that combines visible imaging and computer vision with thermal imaging. It uses a visible-light camera to detect fruit, including small and tiny tomatoes, and a FLIR thermal camera alongside it.

The approach is to align and overlay the visible and thermal imagery, then use the coordinates of a detection in the visible image to read the corresponding thermal measurement for that same piece of fruit — a surface temperature, not an internal one. Pairing “what and where” (from detection) with “how warm at the surface” (from the thermal image) could, in principle, give growers additional information about crop condition and maturity. The detection side is exactly the kind of small-object problem described above: it depends on having enough resolution and image quality on each fruit.

This is an exploration of the technique. It makes no claim of validated agronomic outcomes, detection accuracy, commercial deployment, or improvements in yield or revenue; those would require evidence that does not yet exist. The thermal side of the same work — and a concept diagram of how visible detection and thermal measurement fit together — is in Thermal Imaging.

Sources & further reading

The concepts above — image formation, classical processing, learned models, detection and their failure modes — are standard computer-vision material. For authoritative, deeper treatment:

Szeliski, R., Computer Vision: Algorithms and Applications, 2nd ed., Springer, 2022 (freely available online) — a broad, current reference.
Gonzalez, R. C. & Woods, R. E., Digital Image Processing, Pearson — foundational image-processing theory.
Goodfellow, I., Bengio, Y. & Courville, A., Deep Learning, MIT Press, 2016 (freely available online) — the learned-model foundations.
Prince, S. J. D., Computer Vision: Models, Learning, and Inference, Cambridge University Press, 2012 — a rigorous treatment of the modelling.