Search¶
Ferris provides some higher-level utilities on top of App Engine’s Search API. Notably ferris offers automatic translation of models into search documents via index_entity() and the Searchable behavior as well as a highler-level query interface via search().
Indexing models¶
You can easily add any entity to the full-text search index via index_entity().
- ferris3.search.index_entity(instance, index, only=None, exclude=None, extra_converters=None, indexer=None, callback=None)[source]¶
Adds an Model instance into full-text search indexes.
Parameters: - instance – an instance of ndb.Model
- only (list(string)) – If provided, will only index these fields
- exclude (list(string)) – If provided, will not index any of these fields
- extra_converters (dict) – Extra map of property names or types to converter functions.
- indexer – A function that transforms properties into search index fields.
- callback – A function that will recieve (instance, fields). Fields is a map of property names to search. Field instances generated by the indexer the callback can modify this dictionary to change how the item is indexed.
This is usually done in Model.after_put, for example:
def after_put(self): index(self)
Note
This function can not automatically index Computed, Key, or Blob properties. Use the search_callback to implement indexing for these fields.
Likewise, you can remove an entity via unindex_entity(). Note that this method only requires the key of the entity.
- ferris3.search.unindex_entity(instance_or_key, index=None)[source]¶
Removes a document from the full-text search.
This is usually done in Model.after_delete, for example:
@classmethod def after_delete(cls, key): unindex(key)
Automatic index management¶
The Searchable behavior will automatically add entities to the search index when saved and remove them when deleted:
from ferris import Model
from ferris.behaviors import searchable
class Post(Model):
class Meta:
behaviors = (searchable.Searchable,)
title = ndb.StringProperty()
context = ndb.TextProperty()
- class ferris3.search.Searchable(Model)[source]¶
Automatically indexes models during after_put into the App Engine Text Search API.
This behavior can be configured using the Meta class:
class Meta:
behaviors = (searchable.Searchable,)
search_index = ('global', 'searchable:Post')
search_exclude = ('thumbnail', 'likes')
- SearchableMeta.search_index¶
Which search index to add the entity’s data to. By default this is searchable:[Model]. You can set it to a list or tuple to add the entity data to muliple indexes
- SearchableMeta.search_fields¶
A list or tuple of field names to use when indexing. If not specified, all fields will be used.
- SearchableMeta.search_exclude¶
A list or tuple of field names to exclude when indexing.
- SearchableMeta.search_callback¶
A callback passed to index_entity(). This can be used to index additional fields:
from google.appengine.ext import ndb, search from ferris.behaviors.searchable import Searchable class Post(Model): class Meta: behaviors = (Searchable,) @static_method def search_callback(instance, fields): category = instance.category.get() fields.append( search.TextField( name="category", value=category.title ) ) title = ndb.StringProperty() category = ndb.KeyProperty(Category)
Performing searches¶
Ferris provides a slighly more pythonic wrapper for searching.
- ferris3.search.search(index, query, sort=None, sort_default_values=None, limit=None, page_token=None, ids_only=True, options=None, per_document_cursor=False)[source]¶
Searches an index with the given query and returns a list of document ids or search documents.
To get the full search document pass ids_only = False.Parameters: - index – The name of the index to search.
- query – Query string as described in the App Engine documentation.
- sort – A sort string, can be "field_name" for ascending or "-field_name" for descending. Can also be a list of sorts, such as ["price", "-rating"].
- sort_default_values – The default value to use for sorting if there is no value in the document.
- limit – Maximum number of results to return.
- page_token – Cursor used to get a particular page of results.
- ids_only – By default, this only returns document ids as the most common use case is to get the entity or database entries associated with the document. Pass False here to get the complete document.
- options – Advanced options that are passed directly to index.search. See query options.
- per_document_cursor – Whether to include a cursor for every document or not.
Returns: a tuple of (items, error, next_page_token)
Examples:
# Search all pages for "policies" results, error, next_page_token = search('searchable:Page', 'policies', limit=20) # Search all products for "rake" sorted by price descending results, error, next_page_token = search('searchable:Page', 'rake', sort='-price', limit=20)
Transforming results into entities¶
Search results alone are rarely what you want. Typically, we want to get the actual datastore entity we indexed. This is straightforward with to_entities().
- ferris3.search.to_entities(results)[source]¶
Transform a list of search results into ndb.Model entities by using the document id as the urlsafe form of the key.
For example:
# Search all pages for "policies"
results, error, next_page_token = search.search('searchable:Page', 'policies', limit=20)
entities = search.to_entities(results)
Using search with endpoints¶
A search method can be added to an endpoint service easily using the building blocks above:
import ferris3
from google.appengine.ext import ndb
class Page(ferris3.Model):
class Meta:
behaviors = (ferris3.search.Searchable,)
title = ndb.StringProperty()
content = ndb.TextProperty()
PageMessage = ferris3.model_message(Page)
PageListMessage = ferris3.list_message(PageMessage)
@ferris3.auto_service
class PagesService(ferris3.Service):
@ferris3.auto_method(returns=PageListMessage)
def search(self, request, query=(str,), page_token=(str,None)):
# Get the index to search, because this is a searchable model it'll be 'searchable:Page'
index = ferris3.search.index_for(Page)
# Perform the query
results, error, next_page_token = ferris3.search.search(index, query, limit=20, page_token=page_token)
# Check for errors
if error:
raise ferris3.BadRequestException("Search error: %s" % data.error)
# Translate to entities
entities = ferris3.search.to_entities(results)
# Translate to list message
msg = ferris3.messages.serialize_list(entities, ListMessage)
# Set page token
msg.next_page_token = next_page_token
return msg
The hvild module has a generic implementation of this called searchable_list().