For example, extract entity values before processing trait properties or vice versa. SearchAssist allows you to develop a custom pipeline for each Workbench configuration to suit your business requirements.<\/p>\n
An indexing stage has properties like stage type, stage name, and applicable conditions. Define the conditions to choose the documents that must be transformed. For example, you can write a condition to consider only the FAQs.<\/p>\n
For best results, click the Train your App icon each time you change an indexing configuration. This ensures the indexing is based on the latest configuration.<\/p>\n
To test the SearchAssist app for select stages, render the other stages inactive with the toggle switch in the top right corner of the page. This will retain those stages but at the same time, they won\u2019t be considered in the indexing configuration. You can enable those inactivated stages at a later time as required.<\/p>\n
Indexing pipeline stages are listed here:<\/p>\n
- \n
- Field Mapping<\/b>\u00a0maps fields in an indexing pipeline document to a target field, sets values, copies values, removes fields, and more.<\/li>\n
- Entity Extraction\u00a0<\/b>uses NLP techniques to identify named entities from the source field.<\/li>\n
- Traits Extraction\u00a0<\/b>extracts specific attributes that search users might express in their conversations.<\/li>\n
- Custom Script<\/b>\u00a0stage allows you to enter customized scripts to perform any field mapping tasks like deleting or renaming fields.<\/li>\n
- Keyword Extraction\u00a0<\/b>automatically detects important words stored in a field.<\/li>\n
- Exclude Document<\/b>\u00a0stage drops all the documents that match the specified condition.<\/li>\n
- Semantic Meaning<\/b>\u00a0is a technique to understand the meaning and interpretation of words, signs, and sentence structure. This stage currently supports web page-related sources only.<\/li>\n<\/ul>\n
<\/span><\/span>Field Mapping<\/span><\/span><\/h2>\n
The Field Mapping stage maps fields in an index pipeline document to a target field. SearchAssist allows you to:<\/p>\n
- \n
- Set a field value, rename a field, copy one field to another, or remove a field.<\/li>\n
- Define a condition for the field mapping stage. The field mapping actions occur on the documents that satisfy the given condition.<\/li>\n
- Re-order or delete a Field Mapping.<\/li>\n
- Simulate to test changes before saving.<\/li>\n<\/ul>\n
Training<\/h4>\n
Click the Train your App\u00a0icon each time you change an index configuration. This builds the index based on the updated configurations.<\/p>\n
<\/span><\/span>
![\"\"](\"https:\/\/multisite.korebots.com\/SearchAssist\/wp-content\/uploads\/sites\/18\/2022\/04\/train-your-app-1-e1652101977222-300x64.png\")
<\/a>Configuring Field Mapping<\/span><\/span><\/h3>\nFollow these steps:<\/p>\n
- \n
- Click the Indices menu tab.<\/li>\n
- In the left pane, click Workbench.<\/li>\n
- On the Workbench page, click the + icon.<\/li>\n
- Click the Stage Type field and select Field Mapping.
![\"\"](\"https:\/\/multisite.korebots.com\/SearchAssist\/wp-content\/uploads\/sites\/18\/2022\/04\/field-mapping-e1652102048993-300x148.png\")
<\/a><\/li>\n- Enter a unique stage name.<\/li>\n
- Change the Condition (Basic or Script) if required. The default setting is Basic.<\/li>\n
- Click Choose field and select an item on the drop-down menu.<\/li>\n
- Click Select Operator and choose an item on the drop-down menu.<\/li>\n
- Click + AND to insert another condition. (optional) SearchAssist considers the documents that satisfy the condition(s) as part of the stage.<\/li>\n
- Click the Action field and select an option on the drop-down menu.\n
- \n
- Set<\/b>\u00a0a value for the Field Name<\/li>\n
- Rename<\/b>\u00a0to a change a label of a Field Name<\/li>\n
- Copy<\/b>\u00a0the Source Field to the Target Field<\/li>\n
- Remove<\/b>\u00a0to delete a Field Name<\/li>\n<\/ul>\n<\/li>\n
- Click Field Name and select an item.<\/li>\n
- Click Save Configuration in the top-right corner.<\/li>\n<\/ol>\n
Running a Simulation<\/h4>\n
Click Simulate to verify configurations. The simulator opens a side panel and displays the source, the number of documents with mapping applied, and the result. You can change the source and number of documents if there is no mention in the conditions. (see below)<\/p>\n
Simulation Example<\/h4>\n
- \n
- Select Set on the Actions drop-down menu.<\/li>\n
- Enter Title in the Field Name field.<\/li>\n
- Enter Heading in the Value field.
\n![\"\"](\"https:\/\/multisite.korebots.com\/SearchAssist\/wp-content\/uploads\/sites\/18\/2022\/04\/field-conditions-e1652102105860-300x164.png\")
<\/a><\/li>\n- Click Simulate to verify if the field value is changed.
![\"\"](\"https:\/\/multisite.korebots.com\/SearchAssist\/wp-content\/uploads\/sites\/18\/2022\/04\/simulator-e1652093518107-300x276.png\")
<\/a><\/li>\n<\/ol>\nConditions for Mapping Fields<\/h4>\n
The condition is specified in the following format:<\/p>\n
- \n
- ctx.fieldtype==value or ctx.fieldtype!=value.<\/li>\n<\/ul>\n
For example, ctx.sys_content_type==\u201cweb\u201d to restrict the mapping to content from a web source.<\/p>\n
<\/p>\n
<\/span><\/span>Entity Extraction<\/span><\/span><\/h2>\n
Entity Extraction is the identification of required information expected from the user as input to accomplish the task. SearchAssist uses NLP techniques to find named entities from the source field.<\/p>\n
The Entity Extraction stage in Indexing Pipeline identifies a set of entities from a source field and saves them in a target field. SearchAssist uses the target field to identify the intention of the search user more accurately.<\/p>\n
SearchAssist allows you to:<\/p>\n
- \n
- Add multiple entities to be extracted from the source field.<\/li>\n
- Define a condition for the entity extraction stage and the entities are extracted only from the documents that satisfy the given condition.<\/li>\n
- Re-order or delete entity extraction rules.<\/li>\n
- Simulate changes before saving.<\/li>\n<\/ul>\n
Training<\/h4>\n
Click the Train your App\u00a0icon each time you change an index configuration. This builds the index based on the updated configurations.<\/p>\n
<\/a><\/h3>\n<\/span><\/span>Configuring Entity Extraction<\/span><\/span><\/h3>\n
Follow these steps:<\/p>\n
- \n
- \n
- \n
- Click the Indices menu tab<\/li>\n
- In the left pane, click Workbench.<\/li>\n
- On the Workbench, click the + icon.<\/li>\n
- Click the Stage Type field and select Entity Extraction.
![\"\"](\"https:\/\/multisite.korebots.com\/SearchAssist\/wp-content\/uploads\/sites\/18\/2022\/04\/entity-extraction-e1652102207965-300x151.png\")
<\/a><\/li>\n- Enter a unique stage name.<\/li>\n
- Change the Condition (Basic or Script) if required. The default setting is Basic.<\/li>\n
- Click Choose field and select an item on the drop-down menu.<\/li>\n
- Click Select Operator and choose an item on the drop-down menu.<\/li>\n
- Click + AND to insert another condition. (optional) SearchAssist considers the documents that satisfy the condition(s) as part of the stage.<\/li>\n<\/ol>\n<\/li>\n<\/ol>\n
Naming an Entity<\/h4>\n
- \n
- Click Source Field and select the field you want to extract the entity from.<\/li>\n
- Click the Target Field and choose where to store the extracted entity. The application creates the target field.<\/li>\n
- Click the Add Entities field and select an option. These are the entity types that need to be identified from the source field. The supported entities are Date, Time, Date-Time, Date Period, URL, Email, Location, City, Country, Color, Company Name, Currency, Person Name, Number, Percentage, Phone Number, Zip Code, Quantity, Address, and Airport.<\/li>\n
- Click Save Configuration in the top-right corner.<\/li>\n<\/ol>\n
Running a Simulation<\/h4>\n
Click Simulate to verify configurations. The simulator opens a side panel and displays the source, the number of documents with mapping applied, and the result. You can change the source and number of documents if there is no mention in the conditions. (see below)<\/p>\n
Conditions to Extract Entities<\/i><\/h5>\n
The condition is specified in the following format:<\/p>\n
- \n
- ctx.fieldtype==value or ctx.fieldtype!=value. Obtain the\u00a0field_name<\/i>\u00a0from the Fields table under Index Configuration.<\/li>\n<\/ul>\n
For example, ctx.sys_content_type==\u201dweb\u201d to restrict the extraction to the content from a web source.<\/p>\n
<\/span><\/span>Traits Extraction<\/span><\/span><\/h2>\n
The Traits Extraction stage in Indexing Pipeline extracts specific attributes or details that search users express in their conversations. SearchAssist allows you to identify traits from a source field and save them in a target field. The target field is used to better detect user intentions.<\/p>\n
SearchAssist allows you to:<\/p>\n
- \n
- Add multiple trait groups to be identified from the source field.<\/li>\n
- Define a condition for the trait extraction stage. SearchAssist extracts traits only from the documents that satisfy the given conditions.<\/li>\n
- Re-order or delete trait extractions.<\/li>\n
- Simulate the changes before saving.<\/li>\n<\/ul>\n
Training<\/h4>\n
Click the Train your App\u00a0icon each time you change an index configuration. This builds the index based on the updated configurations.<\/p>\n
<\/a><\/h3>\n<\/span><\/span>Configuring Traits Extraction<\/span><\/span><\/h3>\n
Follow these steps:<\/p>\n
- \n
- Click the Indices menu tab.<\/li>\n
- In the left pane, click Workbench.<\/li>\n
- On the Workbench, click the + icon.<\/li>\n
- Click the Stage Type field and select Traits Extraction.
![\"\"](\"https:\/\/multisite.korebots.com\/SearchAssist\/wp-content\/uploads\/sites\/18\/2022\/04\/traits-extraction-e1652102383563-300x244.png\")
<\/a><\/li>\n- Enter a unique stage name.<\/li>\n
- Change the Condition (Basic or Script) if required. The default setting is Basic.<\/li>\n
- Click Choose field and select an item on the drop-down menu.<\/li>\n
- Click Select Operator and choose an item on the drop-down menu.<\/li>\n
- Click + AND to insert another condition. (optional) SearchAssist considers the documents that satisfy the condition(s) as part of the stage. To learn more, see Conditions in Extraction.<\/li>\n<\/ol>\n
Naming a Trait<\/h4>\n
- \n
- Click Source Field and select the field you want to extract the entity from.<\/li>\n
- Click the Add Trait Groups field, enter an option and click Enter.<\/li>\n
- Click the Target Field and choose where to store the extracted entity. SearchAssist creates the target field.<\/li>\n
- Click Save Configuration in the top-right corner.<\/li>\n<\/ol>\n
Running a Simulation<\/h4>\n
Click Simulate to verify configurations. The simulator opens a side panel and displays the source, the number of documents with mapping applied, and the result. You can change the source and number of documents if there is no mention in the conditions.<\/p>\n
Conditions in\u00a0 Extracting Traits<\/h4>\n
The condition is specified in the following format:<\/p>\n
- \n
- ctx.fieldtype==value or ctx.fieldtype!=value. Obtain the\u00a0field_name<\/i>\u00a0from the Fields table under Index Configuration.<\/li>\n<\/ul>\n
For example, ctx.sys_content_type==\u201dweb\u201d to restrict the extraction to the content from a web source.<\/p>\n
<\/p>\n
<\/span><\/span>Custom Script<\/span><\/span><\/h2>\n
The Custom Script stage in Indexing Pipeline allows code custom processing. You can write a painless script to perform actions such as adding a new field, deleting a field, setting\/resetting values to a field, copying fields, etc.<\/p>\n
Training<\/h4>\n
Click the Train your App\u00a0icon each time you change an index configuration. This builds the index based on the updated configurations.<\/p>\n
<\/a><\/h3>\n<\/p>\n
<\/span><\/span>Configuring a Custom Script<\/span><\/span><\/h3>\n
Follow these steps:<\/p>\n
- \n
- Click the Indices menu tab.<\/li>\n
- In the left pane, click Workbench.<\/li>\n
- On the Workbench, click the + icon.<\/li>\n
- Click the Stage Type field and select Custom Script.
![\"\"](\"https:\/\/multisite.korebots.com\/SearchAssist\/wp-content\/uploads\/sites\/18\/2022\/04\/custom-script-e1652102481945-300x234.png\")
<\/a><\/li>\n- Enter a unique stage name.<\/li>\n
- Change the Condition (Basic or Script) if required. The default setting is Basic.<\/li>\n
- Click Choose field and select an item on the drop-down menu.<\/li>\n
- Click Select Operator and choose an item on the drop-down menu.<\/li>\n
- Click + AND to insert another condition. (optional) SearchAssist considers the documents that satisfy the condition(s) as part of the stage. See below, \u201cConditions for Running Scripts.\u201d<\/li>\n
- In the <\/>Painless field, enter the custom script.
![\"\"](\"https:\/\/multisite.korebots.com\/SearchAssist\/wp-content\/uploads\/sites\/18\/2022\/04\/myCustom-script-e1652102556180-300x184.png\")
<\/a><\/li>\n- Click Save Configuration in the top-right corner.<\/li>\n<\/ol>\n
Running a Simulation<\/i><\/h4>\n
Click Simulate to verify configurations. The simulator opens a side panel and displays the source, number of documents with mapping applied, and the result. You can change the source and number of documents if there is no mention in the conditions. (see below)<\/p>\n
Conditions for Running Scripts<\/i><\/h4>\n
The condition is specified in the following format: ctx.fieldtype==value or ctx.fieldtype!=value. The\u00a0field_name<\/i>\u00a0can be obtained from the Fields table under Index Configuration.<\/p>\n
For example, ctx.sys_content_type==\u201dweb\u201d to restrict the extraction to the content from a web source.<\/p>\n
<\/span><\/span>Keyword Extraction<\/span><\/span><\/h2>\n
Keyword Extraction is a technique to automatically detect important and relevant words from the text stored in a field.<\/p>\n
The Keyword Extraction stage in Indexing Pipeline identifies a set of keywords from a source field and saves them in a target field. Use the target field to identify the intention of the search user. SearchAssist applies various NLP algorithms to extract keywords.<\/p>\n
SearchAssist allows you to:<\/p>\n
- \n
- Define a condition for the keyword extraction stage. SearchAssist extracts keywords only from the documents that satisfy the given condition.<\/li>\n
- Re-order or delete keyword extractions.<\/li>\n
- Simulate changes before saving.<\/li>\n<\/ul>\n
Training<\/h4>\n
Click the Train your App\u00a0icon each time you change an index configuration. This builds the index based on the updated configurations.<\/p>\n
<\/span><\/span>Configuring Keyword Extraction<\/span><\/span><\/h3>\n
Follow these steps:<\/p>\n
- \n
- Click the Indices menu tab.<\/li>\n
- In the left pane, click Workbench.<\/li>\n
- On the Workbench, click the + icon.<\/li>\n
- Click the Stage Type field and select Keyword Extraction.
![\"\"](\"https:\/\/multisite.korebots.com\/SearchAssist\/wp-content\/uploads\/sites\/18\/2022\/04\/key-word-extraction-e1652102619821-300x221.png\")
<\/a><\/li>\n- Enter a unique stage name.<\/li>\n
- Change the Condition (Basic or Script) if required. The default setting is Basic.<\/li>\n
- Click Choose field and select an item on the drop-down menu.<\/li>\n
- Click Select Operator and choose an item on the drop-down menu.<\/li>\n
- Click + AND to insert another condition. (optional) SearchAssist considers the documents that satisfy the condition(s) as part of the stage. See below, \u201cConditions for Keyword Extraction.\u201d<\/li>\n
- Select the field you want to extract keywords from as Source Field.<\/li>\n
- Name the Target Field where you want to store the extracted keywords. The application creates this field.<\/li>\n
- Click Save Configuration in the top-right corner.<\/li>\n<\/ol>\n
Running a Simulation<\/h4>\n
Click Simulate to verify configurations. The simulator opens a side panel and displays the source, the number of documents with mapping applied, and the result. You can change the source and number of documents if there is no mention in the conditions. (see below)<\/p>\n
Conditions for Keyword Extraction<\/h4>\n
The condition is specified in the following format:<\/p>\n
- \n
- ctx.field_name==value or ctx.field_name!=value.<\/li>\n<\/ul>\n
Obtain the\u00a0field_name<\/i>\u00a0from the Fields table under Index Configuration. For example, ctx.sys_content_type==\u201dweb\u201d to restrict the extraction from the content from a web source.<\/p>\n
<\/span><\/span>Exclude Documents<\/span><\/span><\/h2>\n
The Exclude Document stage in Indexing Pipeline drops all the documents that match the specified conditions. SearchAssist ignores documents in the subsequent stages of the indexing pipeline and leaves them without indexing.<\/p>\n
Training<\/h4>\n
Click the Train your App\u00a0icon each time you change an index configuration. This builds the index based on the updated configurations.<\/p>\n
<\/a><\/h3>\n<\/span><\/span>Configuring an Exclusion<\/span><\/span><\/h3>\n
Follow these steps:<\/p>\n
- \n
- Click the Indices menu tab.<\/li>\n
- In the left pane, click Workbench.<\/li>\n
- On the Workbench, click the + icon.<\/li>\n
- Click the Stage Type field and select Exclude Document.
![\"\"](\"https:\/\/multisite.korebots.com\/SearchAssist\/wp-content\/uploads\/sites\/18\/2022\/04\/exclude-docutment-e1652102688283-300x222.png\")
<\/a><\/li>\n- Enter a unique stage name.<\/li>\n
- Change the Condition (Basic or Script) if required. The default setting is Basic.<\/li>\n
- Click Choose field and select an item on the drop-down menu.<\/li>\n
- Click Select Operator and choose an item on the drop-down menu.<\/li>\n
- Click + AND to insert another condition. (optional) SearchAssist excludes documents that satisfy the condition(s). See below, \u201cConditions for Excluding Docs.\u201d<\/li>\n
- Click Save Configuration in the top-right corner.<\/li>\n<\/ol>\n
Note<\/b>:\u00a0You must add at least one condition at this stage.<\/p>\n
![\"\"](\"https:\/\/multisite.korebots.com\/SearchAssist\/wp-content\/uploads\/sites\/18\/2022\/04\/basic-excluded-e1652102757219-300x138.png\")
<\/a><\/p>\nRunning a Simulation<\/h4>\n
Click Simulate to verify configurations. The simulator opens a side panel and displays the source, the number of documents with mapping applied, and the result. You can change the source and number of documents if there is no mention in the conditions. (see below)<\/p>\n
Conditions for Excluding Docs<\/h4>\n
The condition is specified in the following format:\u00a0 ctx.fieldtype==value or ctx.fieldtype!=value. i.e.,\u00a0 the condition can be a certain field type equals a certain value or not equal to a certain value.<\/p>\n
The value here being file types: FAQs, web. The\u00a0field_name<\/i>\u00a0can be obtained from the Fields table under Index Configuration.<\/p>\n
For example, ctx.sys_content_type==\u201dweb\u201d to restrict the extraction to the content from a web source.<\/p>\n
<\/span><\/span>Semantic Meaning<\/span><\/span><\/h2>\n
Semantic analysis is the technique to understand the meaning and interpretation of words, signs, and sentence structure.<\/p>\n
SearchAssist\u2019s Indexing Pipeline supports a Semantic Meaning stage. This stage uses\u00a0Deep Neural Network<\/i>\u00a0algorithms to create inserts of free text and saves them in a dense vector field. SearchAssist uses these embeddings to rank the documents by semantic relevance. The dense vectors are then indexed in the search and similarity is obtained between the user\u2019s query vector and the indexed content vector.<\/p>\n
Currently, you can apply semantic meanings only on the content from websites. When SearchAssist semantically interprets the source from a natural language perspective, the relevant results get priority over others.<\/p>\n
SearchAssist allows you to:<\/p>\n
- \n
- Add multiple semantics to be analyzed from the source field.<\/li>\n
- Define a condition for the semantic meaning stage. The semantics from only the documents that satisfy the given condition are analyzed.<\/li>\n
- Re-order or delete semantic meaning rules.<\/li>\n
- Simulate the changes before saving them.<\/li>\n<\/ul>\n
Training<\/i><\/h5>\n
Click the Train your App\u00a0icon each time you change an index configuration. This builds the index based on the updated configurations.<\/p>\n
<\/p>\n
<\/span><\/span>Configuring Semantic Meaning<\/span><\/span><\/h3>\n
<\/a><\/h3>\nFollow these steps:<\/p>\n
- ctx.fieldtype==value or ctx.fieldtype!=value. Obtain the\u00a0field_name<\/i>\u00a0from the Fields table under Index Configuration.<\/li>\n<\/ul>\n
- ctx.fieldtype==value or ctx.fieldtype!=value. Obtain the\u00a0field_name<\/i>\u00a0from the Fields table under Index Configuration.<\/li>\n<\/ul>\n
- \n
- Click Simulate to verify if the field value is changed.
- Rename<\/b>\u00a0to a change a label of a Field Name<\/li>\n
- Entity Extraction\u00a0<\/b>uses NLP techniques to identify named entities from the source field.<\/li>\n
![\"\"](\"https:\/\/multisite.korebots.com\/SearchAssist\/wp-content\/uploads\/sites\/18\/2022\/04\/train-your-app-1-e1652101977222.png\")
<\/a>Configuring Field Mapping<\/span>