Handling Response Objects

Full Text Search provides a response object, which contains detailed information on the results of the search. Prior to the search, a facet can be add to the response object, and information on aggregations thereby returned.

Response-Object Structure

A Full Text Search Response Object is itself composed of multiple child-objects, each of which provides important information. Each child-object is listed and explained below.

Status

The status object includes the number of successful and failed pindex queries.

We recommend that the status object be checked for failures, as a preferred alternative to relying on HTTP response codes alone. For example,FTS returns an HTTP 200 response in case of pindex failures or timeouts (not consistency timeouts). This is done so that you can choose to accept partial results in an application. However, this also means FTS returns an HTTP 200 response even when ALL pindexes fail.

Refer to the following table for more information about the possible status messages.

Table 1. Response Object Status Values
Status Description

HTTP 400

Returned when an error is detected when validating the inputs. The text error message accompanying the status describes the problem.

HTTP 412

Returned when the consistency requirements are not satisfied within the specified timeout. The JSON structure accompanying the status provides information about the current consistency levels.

HTTP 200

Returned when an error is detected during query execution. The errors may be contained in the JSON section named "status".

Total

The total field returns an integer representing the total number of pindexes queried. This value varies depending on how your index is configured, the platform you are running on, and whether you are querying an index or an index alias. Each index has a setting for number of vBuckets per pindex. For example, when running on a cluster with 1024 vBuckets and with an index setting of 171 vBuckets per pindex, the total number of pindexes is 6. When you query an index alias, total is the sum of all the pindexes in each index that is included in the definition of the index alias.

Successful

This field returns the total number of pindexes successfully queried. This integer value can range from 0 to the total number of pindexes.

Example Status: Successful
 "status": {
     "total": 6,
     "failed": 0,
     "successful": 6
},
Failed

This field returns the total number of pindexes with failed queries. This integer value can range from 0 to the total number of pindexes. If the number is greater than 0, the response object contains an errors array.

Errors

This returns an array of error messages as reported by individual pindexes. This array only appears if the field "failed" is greater than 0.

Example Status: Failure
"status": {
    "total":6,
    "failed":3,
    "successful":6,
    "errors":{
        "IndexClient - http://127.0.0.1:9201/api/pindex/beer-search_447fd6df2d6f4b54_0a44bddb":"context deadline exceeded",
         "IndexClient - http://127.0.0.1:9201/api/pindex/beer-search_447fd6df2d6f4b54_24e7ea2d":"context deadline exceeded",
         ...
    }
}

Understanding the Query Response Status

This section explains the contents of a typical query response status.

Request

The request object stores a copy of the query that was executed and other details from the request that generated this response object: for example the number of results requested (size), the offset, highlighting, and so on.

Query

Represents the actual query that was executed.

Size

Represents the number of results requested. The default size is 10. If the size is set to 0, the query returns the count of matches with no other results.

From

Results of a query are returned in the descending score order. This field specifies the offset, starting at 0 (default) for the first match, to return the results.

Highlight

This object indicates whether highlighting was requested. The highlight object contains two fields:

  • style - Optional field that specifies the name of the highlighter, which can be "html", "ansi", or a custom highlighter.

  • fields - Specifies an array of field names to which highlighting is restricted.

Fields

This object contains an array of field names which must be loaded and returned with the query response. If you store a field and want it returned with the result, you must name the field here. The field names are specified as string values. While there is no field name pattern matching available, you can use an asterisk "*" to specify that all the fields must be loaded and returned with the response.

Facets

This object indicates whether faceting was requested and when requested, returns the facets. See below for detailed information on facets.

"facets": {
    "type": {
        "size": 5,
        "field": "type"
    }
}
Explain

This Boolean field when set to true prints a verbose response with full scoring information.

Hits

Hits returns an array containing the matches for the executed query. The length of the array is equal to or less than the size specified in the request.

Index

The unique ID of the pindex. The index name always begins with a string.

ID

The document ID that matched.

Score

The document score.

Locations

This object contains field names where matches were found. The "Locations" object depends on the term vectors being stored; if term vectors are not stored, locations are not returned in the result object.

{Field Name}

Lists the field names where the match was found. These fields are scoped so that "description: american" searches for "american" scoped to the "description" field. In the example below, there are two fields named "description" and "name".

{Term Found}

A name value pair whose name is the name of the term that was found and whose value is an array on objects representing the vector information that describes the position of the matched term in the field. This value is only present if the term vectors are calculated. For each match, the object contains the position (pos), start, end, and array positions (array_positions).

Sample Locations Fragment
"locations": {
    "reviews.content": {
        "light": [
            {
                "pos": 277,
                "start": 1451,
                "end": 1456,
                "array_positions": [
                    0
                ]
           },
           {
               "pos": 247,
               "start": 1321,
               "end": 1326,
               "array_positions": [
                   3
               ]
           }
       ]
   }
},
Fragments

These objects, also known as snippets, contain field names that contain an array of one or more text strings. The text strings contain the "<mark>" tags surrounding the term that was matched in order to render highlighting.

Fields

This object returns the value of the field that was matched. However, unlike the Fragments field, this does not have any tags to render highlighting.

Sort

This field contains an array of one or more values that were used to sort the search results. Documents that don’t have a value in the index for a particular field used in sorting will return a series of non-printable Unicode characters: \ufffd\ufffd\ufffd\ufffd\ufffd\ufffd\ufffd\ufffd\ufffd\ufffd.

Total_hits

Total hits represents the total number of matches for this result. It can be any integer starting from 0.

Max_score

Max score represents the highest score of all documents for this query.

Took

Time taken to complete the query.

"total_hits": 56,
"max_score": 0.8041158525040355,
"took": 1449005,

Response Headers

Response headers can contain the following information:

Table 2. Response Headers
Code Example Valid Return Codes

Status

HTTP/1.1 200 OK

200 OK

400 Bad Request, returned if the query is invalid due to malformed JSON or invalid consistency request.

412 if timeout occurs before the requested consistency requirements are met.

Cache-Control

no-cache

Content-Type

application/json; version=1.0.0

The API version information is included in this field unless the response is HTTP 400, in which case the response will be "text/plain: charset=utf-8"

Date

Tue, 22 Mar 2016 19:28:57 GMT

Date of the response

Transfer-Encoding

chunked

X-Content-Type-Options

nosniff

Value "https://blogs.msdn.microsoft.com/ie/2008/09/02/ie8-security-part-vi-beta-2-update/[nosniff^]" is returned in case of a bad request (400 or 412) in order to deter driveby downloads.

Query Counts

All queries return a result count. To get just the count of documents that match a particular query without returning documents or ids, execute the query as usual but specify size "0" to return no results, as in the following example:

curl -X POST -H "Content-Type: application/json" \
http://127.0.0.1:8094/api/index/beer-idx/query -d \
'{
    "indexName": "beer-idx",
    "size": 0,
    "from": 0,
    "explain": true,
    "highlight": {},
    "query": {
        "boost": 1,
        "query": "geo.accuracy:rooftop"
    },
    "fields": [
        "*"
    ],
    "ctl": {
        "consistency": {
            "level": "",
            "vectors": {}
        },
        "timeout": 0
    }
}'

You can get a count of entries in an index overall by using the REST API:

http://localhost:8094/api/index/beer-idx/count

Search Facets

Facets are aggregate information collected on a particular result set. So, you have to already have a search in mind, and then you collect additional facet information along with it. All of the facet examples below are for the query "water" on the beer-sample dataset.

FTS supports 3 types of facet:

  • Term Facet - A term facet counts up how many of the matching documents have a particular term in a particular field. Most of the time this only makes sense for relatively low cardinality fields, like a type or tags. It would not make sense to use it on a unique field like an ID.

  • Numeric Range Facet - A numeric range facet works by the user defining their own buckets (numeric ranges). The facet then counts how many of the matching documents fall into a particular bucket for a particular field.

  • Date Range Facet - same as numeric, but on dates instead of numbers. Full text search and Bleve expect dates to be in the format specified by RFC-3339, which is a specific profile of ISO-8601 that is more restrictive.

Most of the time, when building a term facet you want to use the keyword analyzer. Otherwise multi-term values get tokenized and the results are not what you expect.

Examples

  • Term Facet - computes facet on the type field which has 2 values: beer and brewery.

    curl -X POST -H "Content-Type: application/json" \
    http://localhost:8094/api/index/bix/query -d \
    '{
        "size": 10,
        "query": {
            "boost": 1,
            "query": "water"
         },
        "facets": {
             "type": {
                 "size": 5,
                 "field": "type"
             }
        }
    }'

    The result snippet below only shows the facet section for clarity. Run the curl command to see the HTTP response containing the full results.

    "facets": {
        "type": {
            "field": "type",
            "total": 91,
            "missing": 0,
            "other": 0,
            "terms": [
                {
                    "term": "beer",
                    "count": 70
                },
                {
                    "term": "brewery",
                    "count": 21
                }
            ]
        }
    }
  • Numeric Range Facet - computes facet on the abv field with 2 buckets describing high (greater than 7) and low (less than 7).

    curl -X POST -H "Content-Type: application/json" \
    http://localhost:8094/api/index/bix/query -d \
    '{
        "size": 10,
        "query": {
            "boost": 1,
            "query": "water"
        },
        "facets": {
            "abv": {
                "size": 5,
                "field": "abv",
                "numeric_ranges": [
                    {
                        "name": "high",
                        "min": 7
                    },
                    {
                        "name": "low",
                        "max": 7
                    }
                 ]
            }
        }
    }'

    Results:

    facets": {
        "abv": {
            "field": "abv",
            "total": 70,
            "missing": 21,
            "other": 0,
            "numeric_ranges": [
                {
                    "name": "high",
                    "min": 7,
                    "count": 13
                },
                {
                    "name": "low",
                    "max": 7,
                    "count": 57
                }
            ]
        }
    }