{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "a1000001",
   "metadata": {},
   "source": [
    "# Vector Search Quickstart\n",
    "\n",
    "ApertureDB stores vectors as Descriptors within a named DescriptorSet. The set defines their dimensions, index engine, and distance metric. `FindDescriptor` compares a query vector with the stored vectors and returns nearby matches.\n",
    "\n",
    "This example uses small synthetic vectors to show ingestion, KNN search, and the Python SDK wrapper without downloading an embedding model.\n",
    "\n",
    "The examples require Python 3.10 or later, an ApertureDB connection profile or environment credentials, and permission to create, read, update, and delete objects. A unique `documentation_run` property separates the example data from other records. Cleanup uses that value, so keep the setup variables available until cleanup completes.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a1000002",
   "metadata": {},
   "source": [
    "## Connect to ApertureDB\n",
    "\n",
    "**ApertureDB Cloud**\n",
    "The SDK connects with an API key from your [Cloud instance](https://cloud.aperturedata.io). Obtain the key from **Connect > Generate API Key** and add it to a `.env` file in this directory:\n",
    "```\n",
    "APERTUREDB_KEY=your_key_here\n",
    "```\n",
    "\n",
    "**Local evaluation:** follow the [deployment guide](https://docs.aperturedata.io/administration/deployment/DockerCompose), then configure your connection profile.\n",
    "\n",
    "\n",
    "See [client configuration options](https://docs.aperturedata.io/Setup/client/configuration) for all connection methods and [server setup options](https://docs.aperturedata.io/category/setup-server) for deployment choices."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "a1000003",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-10-14T19:38:10.238322Z",
     "iopub.status.busy": "2026-10-14T19:38:10.238219Z",
     "iopub.status.idle": "2026-10-14T19:38:14.619726Z",
     "shell.execute_reply": "2026-10-14T19:38:14.619076Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Note: you may need to restart the kernel to use updated packages.\n"
     ]
    }
   ],
   "source": [
    "%pip install --quiet \"aperturedb==0.5.1\" \"python-dotenv==1.1.1\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "a1000004",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-10-14T19:38:14.621080Z",
     "iopub.status.busy": "2026-10-14T19:38:14.620959Z",
     "iopub.status.idle": "2026-10-14T19:38:14.626404Z",
     "shell.execute_reply": "2026-10-14T19:38:14.626062Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Option A: ApertureDB Cloud\n",
    "from dotenv import load_dotenv\n",
    "load_dotenv()  # loads APERTUREDB_KEY from .env into the environment"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "a1000005",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-10-14T19:38:14.627608Z",
     "iopub.status.busy": "2026-10-14T19:38:14.627528Z",
     "iopub.status.idle": "2026-10-14T19:38:14.629018Z",
     "shell.execute_reply": "2026-10-14T19:38:14.628656Z"
    }
   },
   "outputs": [],
   "source": [
    "# For local evaluation, configure the profile from your versioned deployment package.\n",
    "# https://docs.aperturedata.io/administration/deployment/DockerCompose\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "a1000006",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-10-14T19:38:14.630099Z",
     "iopub.status.busy": "2026-10-14T19:38:14.630018Z",
     "iopub.status.idle": "2026-10-14T19:38:14.760742Z",
     "shell.execute_reply": "2026-10-14T19:38:14.760411Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[\n",
      "    {\n",
      "        \"GetStatus\": {\n",
      "            \"info\": \"OK\",\n",
      "            \"status\": 0,\n",
      "            \"system\": \"ApertureDB\",\n",
      "            \"version\": \"0.19.15\"\n",
      "        }\n",
      "    }\n",
      "]\n"
     ]
    }
   ],
   "source": [
    "from aperturedb.CommonLibrary import create_connector\n",
    "\n",
    "client = create_connector()\n",
    "\n",
    "import uuid\n",
    "\n",
    "documentation_run = uuid.uuid4().hex\n",
    "ingredient_class = \"CookbookIngredient_\" + documentation_run\n",
    "connection_class = \"CookbookIngredientLink_\" + documentation_run\n",
    "\n",
    "def checked_query(query, input_blobs=None):\n",
    "    response, output_blobs = client.query(query, input_blobs or [])\n",
    "    assert isinstance(response, list) and len(response) == len(query), response\n",
    "    for command in response:\n",
    "        result = next(iter(command.values()))\n",
    "        assert result.get(\"status\") == 0, response\n",
    "    return response, output_blobs\n",
    "\n",
    "response, _ = checked_query([{\"GetStatus\": {}}])\n",
    "client.print_last_response()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a1000007",
   "metadata": {},
   "source": [
    "## Create a Vector Index\n",
    "\n",
    "A [DescriptorSet](https://docs.aperturedata.io/query_language/Reference/descriptor_commands/desc_set_commands/AddDescriptorSet) is a named, indexed collection of vectors. All vectors in a set must have the same number of dimensions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "a1000008",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-10-14T19:38:14.762165Z",
     "iopub.status.busy": "2026-10-14T19:38:14.762053Z",
     "iopub.status.idle": "2026-10-14T19:38:14.769515Z",
     "shell.execute_reply": "2026-10-14T19:38:14.769039Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[\n",
      "    {\n",
      "        \"AddDescriptorSet\": {\n",
      "            \"status\": 0\n",
      "        }\n",
      "    }\n",
      "]\n"
     ]
    }
   ],
   "source": [
    "import uuid\n",
    "SET_NAME = \"recipe_search_\" + uuid.uuid4().hex[:12]\n",
    "\n",
    "checked_query([{\n",
    "    \"AddDescriptorSet\": {\n",
    "        \"name\":       SET_NAME,\n",
    "        \"dimensions\": 4,           # use 384, 512, 1024, etc. for real models\n",
    "        \"engine\":     \"Flat\", # exact search; use HNSW for large-scale ANN\n",
    "        \"metric\":     \"CS\",        # cosine similarity; or \"L2\" for Euclidean\n",
    "    }\n",
    "}])\n",
    "client.print_last_response()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a1000009",
   "metadata": {},
   "source": [
    "## Add Vectors\n",
    "\n",
    "Each [Descriptor](https://docs.aperturedata.io/query_language/Reference/descriptor_commands/desc_commands/AddDescriptor) is a float32 vector plus optional metadata properties. The vector is passed as a binary blob."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "a1000010",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-10-14T19:38:14.770682Z",
     "iopub.status.busy": "2026-10-14T19:38:14.770595Z",
     "iopub.status.idle": "2026-10-14T19:38:14.820113Z",
     "shell.execute_reply": "2026-10-14T19:38:14.818755Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Added 5 descriptors\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "\n",
    "dishes = [\n",
    "    {\"name\": \"Butter Chicken\",  \"cuisine\": \"Indian\",    \"vec\": [0.9, 0.1, 0.8, 0.2]},\n",
    "    {\"name\": \"Rajma Chawal\",    \"cuisine\": \"Indian\",    \"vec\": [0.8, 0.2, 0.9, 0.1]},\n",
    "    {\"name\": \"Ramen\",           \"cuisine\": \"Japanese\",  \"vec\": [0.1, 0.9, 0.2, 0.8]},\n",
    "    {\"name\": \"Sushi\",           \"cuisine\": \"Japanese\",  \"vec\": [0.2, 0.8, 0.1, 0.9]},\n",
    "    {\"name\": \"Focaccia\",        \"cuisine\": \"Italian\",   \"vec\": [0.5, 0.5, 0.6, 0.4]},\n",
    "]\n",
    "\n",
    "for dish in dishes:\n",
    "    vec = np.array(dish[\"vec\"], dtype=\"float32\")\n",
    "    checked_query([{\n",
    "        \"AddDescriptor\": {\n",
    "            \"set\": SET_NAME,\n",
    "            \"properties\": {\n",
    "                           'documentation_run': documentation_run,\"name\": dish[\"name\"], \"cuisine\": dish[\"cuisine\"]},\n",
    "        }\n",
    "    }], [vec.tobytes()])\n",
    "\n",
    "print(f\"Added {len(dishes)} descriptors\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a1000011",
   "metadata": {},
   "source": [
    "## KNN Search\n",
    "\n",
    "[FindDescriptor](https://docs.aperturedata.io/query_language/Reference/descriptor_commands/desc_commands/FindDescriptor) accepts a query vector and returns up to `k_neighbors` matches using the selected engine and distance metric. Approximate engines may return a different set of neighbors from an exact search.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "a1000012",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-10-14T19:38:14.822899Z",
     "iopub.status.busy": "2026-10-14T19:38:14.822730Z",
     "iopub.status.idle": "2026-10-14T19:38:14.828074Z",
     "shell.execute_reply": "2026-10-14T19:38:14.827605Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[\n",
      "    {\n",
      "        \"FindDescriptor\": {\n",
      "            \"entities\": [\n",
      "                {\n",
      "                    \"_distance\": 0.9966610670089722,\n",
      "                    \"_set_name\": \"recipe_search_bc0cb3c3694f\",\n",
      "                    \"_uniqueid\": \"3.1.800\",\n",
      "                    \"cuisine\": \"Indian\",\n",
      "                    \"documentation_run\": \"747cfc3aed594adb965205cc7a3f6eca\",\n",
      "                    \"name\": \"Rajma Chawal\"\n",
      "                },\n",
      "                {\n",
      "                    \"_distance\": 0.9966610074043274,\n",
      "                    \"_set_name\": \"recipe_search_bc0cb3c3694f\",\n",
      "                    \"_uniqueid\": \"3.0.780\",\n",
      "                    \"cuisine\": \"Indian\",\n",
      "                    \"documentation_run\": \"747cfc3aed594adb965205cc7a3f6eca\",\n",
      "                    \"name\": \"Butter Chicken\"\n",
      "                },\n",
      "                {\n",
      "                    \"_distance\": 0.8679417371749878,\n",
      "                    \"_set_name\": \"recipe_search_bc0cb3c3694f\",\n",
      "                    \"_uniqueid\": \"3.4.860\",\n",
      "                    \"cuisine\": \"Italian\",\n",
      "                    \"documentation_run\": \"747cfc3aed594adb965205cc7a3f6eca\",\n",
      "                    \"name\": \"Focaccia\"\n",
      "                }\n",
      "            ],\n",
      "            \"returned\": 3,\n",
      "            \"status\": 0\n",
      "        }\n",
      "    }\n",
      "]\n"
     ]
    }
   ],
   "source": [
    "query_vec = np.array([0.85, 0.15, 0.85, 0.15], dtype=\"float32\")  # close to Indian dishes\n",
    "\n",
    "response, _ = checked_query([{\n",
    "    \"FindDescriptor\": {\n",
    "        'constraints': ['$documentation_run', '==', documentation_run],\n",
    "        \"set\":         SET_NAME,\n",
    "        \"k_neighbors\": 3,\n",
    "        \"distances\":   True,\n",
    "        \"results\":     {\"all_properties\": True},\n",
    "    }\n",
    "}], [query_vec.tobytes()])\n",
    "\n",
    "client.print_last_response()\n",
    "\n",
    "assert response[0][\"FindDescriptor\"][\"returned\"] == 3\n",
    "assert {item[\"name\"] for item in response[0][\"FindDescriptor\"][\"entities\"][:2]} == {\"Butter Chicken\", \"Rajma Chawal\"}\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a1000013",
   "metadata": {},
   "source": [
    "## Python SDK: `Descriptors` Wrapper\n",
    "\n",
    "The [`Descriptors`](https://docs.aperturedata.io/python_sdk/object_wrappers/Descriptors) class in the Python SDK wraps the query language and adds reranking with MMR."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "a1000014",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-10-14T19:38:14.828989Z",
     "iopub.status.busy": "2026-10-14T19:38:14.828860Z",
     "iopub.status.idle": "2026-10-14T19:38:15.324408Z",
     "shell.execute_reply": "2026-10-14T19:38:15.323870Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "find_similar:\n",
      "  Rajma Chawal         distance=0.9967\n",
      "  Butter Chicken       distance=0.9967\n",
      "  Focaccia             distance=0.8679\n",
      "\n",
      "find_similar_mmr (diversified):\n",
      "  Rajma Chawal         cuisine=Indian\n",
      "  Butter Chicken       cuisine=Indian\n",
      "  Focaccia             cuisine=Italian\n"
     ]
    }
   ],
   "source": [
    "from aperturedb.Descriptors import Descriptors\n",
    "\n",
    "descriptors = Descriptors(client)\n",
    "\n",
    "# Basic similarity search — distances available\n",
    "descriptors.find_similar(\n",
    "    set=SET_NAME,\n",
    "    vector=query_vec,\n",
    "    k_neighbors=3,\n",
    "    distances=True,\n",
    ")\n",
    "print(\"find_similar:\")\n",
    "for r in descriptors.response:\n",
    "    print(f\"  {r['name']:<20} distance={r['_distance']:.4f}\")\n",
    "\n",
    "print()\n",
    "\n",
    "# MMR: diversify results (avoids near-duplicates)\n",
    "# Note: find_similar_mmr uses blobs internally for reranking;\n",
    "# _distance is not available in the output.\n",
    "descriptors.find_similar_mmr(\n",
    "    set=SET_NAME,\n",
    "    vector=query_vec,\n",
    "    k_neighbors=3,\n",
    "    fetch_k=5,\n",
    "    lambda_mult=0.5,   # 0.0 = max diversity, 1.0 = similarity only\n",
    ")\n",
    "print(\"find_similar_mmr (diversified):\")\n",
    "for r in descriptors.response:\n",
    "    print(f\"  {r['name']:<20} cuisine={r['cuisine']}\")\n",
    "\n",
    "assert len(descriptors.response) == 3\n",
    "assert len({item[\"name\"] for item in descriptors.response}) == 3\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a1000015",
   "metadata": {},
   "source": [
    "## Cleanup"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "a1000016",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-10-14T19:38:15.325559Z",
     "iopub.status.busy": "2026-10-14T19:38:15.325408Z",
     "iopub.status.idle": "2026-10-14T19:38:15.610395Z",
     "shell.execute_reply": "2026-10-14T19:38:15.610009Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[\n",
      "    {\n",
      "        \"DeleteDescriptorSet\": {\n",
      "            \"count\": 1,\n",
      "            \"status\": 0\n",
      "        }\n",
      "    }\n",
      "]\n"
     ]
    }
   ],
   "source": [
    "checked_query([{\"DeleteDescriptorSet\": {\"with_name\": SET_NAME}}])\n",
    "client.print_last_response()\n",
    "remaining, _ = checked_query([{\"FindDescriptorSet\": {\"with_name\": SET_NAME, \"results\": {\"all_properties\": True}}}])\n",
    "assert remaining[0][\"FindDescriptorSet\"][\"returned\"] == 0\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a1000017",
   "metadata": {},
   "source": [
    "## Next Steps\n",
    "\n",
    "The following examples generate embeddings from text, images, and video, or cover related storage and ingestion tasks:\n",
    "\n",
    "| Data type | Notebook |\n",
    "|-----------|----------|\n",
    "| Text / documents | [Recipe Text Search](https://docs.aperturedata.io/HowToGuides/start/TextSearch) — sentence-transformer embeddings of dish descriptions |\n",
    "| PDF | [Work with PDFs](https://docs.aperturedata.io/HowToGuides/start/PDF) — chunk, embed, and search a PDF |\n",
    "| Images | [Image Vector Search](https://docs.aperturedata.io/HowToGuides/start/ImageVectorSearch) — CLIP embeddings and text-to-image search |\n",
    "| Video frames | [Video Vector Search](https://docs.aperturedata.io/HowToGuides/start/VideoVectorSearch) — CLIP embeddings and text-to-frame search |\n",
    "| Audio storage | [Work with Audio](https://docs.aperturedata.io/HowToGuides/start/AudioVectorSearch) — store audio and query its metadata |\n",
    "| Bulk loading | [Bulk Embeddings](https://docs.aperturedata.io/HowToGuides/start/BulkEmbeddings) — concurrent ingestion with ParallelLoader |\n",
    "| Hybrid search | [Hybrid Search](https://docs.aperturedata.io/HowToGuides/start/HybridSearch) — combine vector similarity with metadata filters |\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}