Adding Custom Embeddings
Add custom embeddings to your uploads
Aquarium uses neural network embeddings to enable features like clustering and similarity search. They can be thought of as lists of numbers that represent the essential visual qualities of an image. By default, Aquarium will try to compute embeddings for you by using our standard neural network.
When uploading custom embeddings, frame and crop level embeddings must be the same.
You may also wish to provide your own embeddings, especially if you have an unusual data domain. Our python library makes it easy to attach your own values instead:
# Embedding vectors are expected to by python lists of float values.
# You can provide both frame-level embeddings (for the entire image or scene)
# and crop/label-level embeddings (for that specific object)
frame.add_frame_embedding(embedding=[1.0,2.0,3.0, ...])
for label_id, label_embedding in label_embeddings.items():
frame.add_crop_embedding(label_id=label_id, embedding=label_embedding)
# Similar APIs exist for inferences
inference_frame.add_frame_embedding(embedding=[1.0,2.0,3.0, ...])
for inf_id, inf_embedding in inference_embeddings.items():
inference_frame.add_crop_embedding(label_id=inf_id, embedding=inf_embedding)
If you chose a data sharing scheme that doesn't allow Aquarium to access your data, then you'll have to provide your own embeddings to enable certain features. Check out our full page on embeddings for some sample code, along with guidance for using your own models.
Copy link