Visual Patent Search Magic with BlobBot

In a previous post, I mentioned that I’ve been developing / exploring some new search tools, machine learning and otherwise, as I find time. Generally, I’ve been having some pretty decent success.

Naturally, I was rather oblique about what actually works and what I actually use, but I don’t mind sharing a few things that didn’t work or which were stepping stones to more fruitful systems. One of those less fruitful stepping stones was a prototype image search based on affine-invariant classifiers.

That’s a fancy way of saying that I looked for squished splatter blobs that looked like other squished splatter blobs. Hence, BlobBot.

The interface looked like the below:

An Elegant Search Interface

In the top left you drew polygonal search shapes, the system found similar shapes in the right column, and displayed the top match in the dark grey center. To wit, after clicking up a polygon like so:

Click…click…click…

and merging a few such polygonal shapes, the system would begin hunting for similar shapes. Here, e.g., I drew a triangular, circular looking thing and the system found a similar looking robot end effector:

Rembrandt has nothing on this

Note the vertical list on the right – this end effector was the “top” match, with the next best matches in descending order. As indicated, the system had an inordinate fondness for flow and block diagrams.

Here I drew an “upside down stick in mud” looking thing and the system returned a corresponding “upside down stick in the mud” manipulator.

A match made in heaven . . . or just in a multidimensional search space . . .

Actually, I’m pretty sure what matched here was a 180-degree rotated version of the search target, so really a “right side up stick in the mud.”

The thing was even good about finding some pretty complicated items like this . . . whatever this is.

The Thing

But it’s not very useful for flow charts or block diagrams since they all kind of look the same.

Not wrong, but not terribly helpful either

Also, as the search set changes, the false positive rate would vary. E.g., if you added some sub-sea references, searching for the triangular circle would still produce that end effector, but it often wasn’t the top hit.

Am I a bad polygon artist or is it a bad system?

Ultimately, for legal purposes, there are better vehicles for finding what you need. In those cases where visual inspection is necessary, you’ve likely already reduced the search space to something manageable.

Still, BlobBot’s memory footprint is pretty small, so I may add it to this website someday so folks can play with it. If you have such an interest, let me know.