Timeout error: Operation 'Search' timed out after 60 seconds

[linogaliana]

I was trying to use a Qdrant database from Python. The database is self-hosted on a kubernetes cluster.

The operation I want to perform is greedy: from a collection of around 1 million points, I want to retrieve all those with a minimum score_threshold. Positive echoes can therefore be very numerous, but this is a one-time operation. Incidentally, to reduce the volume of info returned from Qdrant, I use the with_payload argument to return only the id of documents that meet the criterion.

When setting a low limit (e.g. 1000), everything is fine. However, by increasing the limit, I end up with the following error, either with the python client or the REST API:

Timeout error: Operation \‘Search\’ timed out after 60 seconds

I've tried increasing the timeout parameter but it doesn't seem to make any difference. Is there a solution ?

Here is my code:

Python client

client = QdrantClient(
    url=os.getenv("QDRANT_SERVER_HOST"), api_key=os.getenv("QDRANT_API_KEY"),
    port=None, https=True,
    timeout=1000
)

relevant_docs_from_qdrant = client.query_points(
    collection_name=COLLECTION,
    query = emb_model.embed_query("data"),
    using=EMBEDDING_MODEL,
    with_payload=["metadata.job_id"],
    limit=100000,
    score_threshold=0.5
)

Direct API call

url = f"{os.getenv("QDRANT_SERVER_HOST")}/collections/{COLLECTION}/points/query"
headers = {
    "api-key": os.getenv("QDRANT_API_KEY"),
    "Content-Type": "application/json"
}
payload = {
    "query": emb_model.embed_query("data"),
    "using": EMBEDDING_MODEL,
    "limit": 100000,
    "score_threshold": 0.5,
    "with_payload": ["metadata.job_id"],
    "timeout": 1000
}

logger.info("Launching API query")

response = requests.post(url, headers=headers, json=payload, timeout=1000)
response.content