{"id":1126,"date":"2021-06-09T15:29:11","date_gmt":"2021-06-09T15:29:11","guid":{"rendered":"https:\/\/multisite.korebots.com\/SearchAssist\/?p=1126"},"modified":"2021-06-09T15:29:11","modified_gmt":"2021-06-09T15:29:11","slug":"experiment","status":"publish","type":"post","link":"https:\/\/multisite.korebots.com\/SearchAssist\/analytics\/experiment\/","title":{"rendered":"Define Experiments"},"content":{"rendered":"<section class=\"l-section wpb_row height_auto\"><div class=\"l-section-h i-cf\"><div class=\"g-cols vc_row via_grid cols_1 laptops-cols_inherit tablets-cols_inherit mobiles-cols_1 valign_top type_default stacking_default\"><div class=\"wpb_column vc_column_container\"><div class=\"vc_column-inner\"><div class=\"wpb_text_column\"><div class=\"wpb_wrapper\"><p>Experiments are the best way to test multiple index and search configuration combinations and pick the best one that fulfills your business needs. SearchAssist allows you to create experiments with two or more variants to continuously improve search relevance and user experience.<\/p>\n<\/div><\/div><div class=\"wpb_text_column\"><div class=\"wpb_wrapper\"><h2><span class=\"ez-toc-section\" id=\"Overview\"><\/span><span style=\"font-weight: 400;\">Overview<\/span><span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>Consider the following scenarios:<\/p>\n<p><span style=\"font-weight: 400;\">Scenario 1: You have <a href=\"\/SearchAssist\/manage-index\/index-management\/\" target=\"_blank\" rel=\"noopener noreferrer\">configured <\/a>an index, and tuned <a href=\"\/SearchAssist\/optimize-results\/result-optimization\/\" target=\"_blank\" rel=\"noopener noreferrer\">search configurations<\/a> to optimize search results. You have run it against test data in a controlled environment, but will these settings work with real-time data?<br \/>\n<\/span><\/p>\n<p><span style=\"font-weight: 400;\">Scenario 2: You have deployed your search application and have <a href=\"\/SearchAssist\/analytics\/analytics\/\" target=\"_blank\" rel=\"noopener noreferrer\">analyzed<\/a> its performance. You want to tweak the index or search configuration or both a little. You have cloned the existing configuration and made the necessary changes. How can you ensure that these tweaks would work?<\/span><\/p>\n<p>Using Experiments, you can find out which index\/search configuration is more effective. It helps you:<\/p>\n<ul>\n<li>Create variants (up to 4) using a combination of previously created indices and search configurations.<\/li>\n<li>Run them live by splitting the traffic to each variant.<\/li>\n<li>See which one performs best based on the metrics like clicks and click-through rate.<\/li>\n<\/ul>\n<p>Internally, every search is associated with a unique user identifier. This serves two purposes:<\/p>\n<ul>\n<li>It ensures randomness. The application creates sets of users &#8211; one for each variant. Whenever a new user arrives, they are randomly distributed into either of the variants, based on a hash of their unique user identifier.<\/li>\n<li>It maintains the same distribution. If a user is assigned a variant, they continue with the same variant thus ensuring that the experiment conclusion is reliable.<\/li>\n<\/ul>\n<p>You can further control the experiments by:<\/p>\n<ul>\n<li>Specifying the percent of traffic assigned to each of the variants.<\/li>\n<li>Setting the duration of an experiment.<\/li>\n<\/ul>\n<\/div><\/div><div class=\"w-separator size_small with_line width_default thick_1 style_solid color_border align_center\"><div class=\"w-separator-h\"><\/div><\/div><div class=\"wpb_text_column\"><div class=\"wpb_wrapper\"><h2><span class=\"ez-toc-section\" id=\"Add_Experiments\"><\/span>Add Experiments<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p><span style=\"font-weight: 400;\">To add an experiment, follow the below steps:<\/span><\/p>\n<ol>\n<li style=\"font-weight: 400;\"><span style=\"font-weight: 400;\">Click the <\/span><b>Analytics <\/b><span style=\"font-weight: 400;\">tab on the top and select <\/span><b>Experiments <\/b><span style=\"font-weight: 400;\">from the drop-down list.<\/span><\/li>\n<li style=\"font-weight: 400;\"><span style=\"font-weight: 400;\">On the Experiments page, click <\/span><b>+ Add Experiment<\/b><span style=\"font-weight: 400;\"> on the top-right.<\/span><\/li>\n<li style=\"font-weight: 400;\"><span style=\"font-weight: 400;\">On the <\/span><b>New Experiment<\/b><span style=\"font-weight: 400;\"> dialog box, enter a name in the <\/span><b>Name of Experiment <\/b><span style=\"font-weight: 400;\">field.<\/span><\/li>\n<li style=\"font-weight: 400;\"><span style=\"font-weight: 400;\">Click <\/span><b>+ Add Variant<\/b><span style=\"font-weight: 400;\">.<\/span><\/li>\n<li style=\"font-weight: 400;\"><span style=\"font-weight: 400;\">Enter a name in the <\/span><b>Variant Name<\/b><span style=\"font-weight: 400;\"> field.<\/span><\/li>\n<li style=\"font-weight: 400;\"><span style=\"font-weight: 400;\">Select an index configuration <\/span><span style=\"font-weight: 400;\">from the <\/span><b>Index<\/b><span style=\"font-weight: 400;\"> drop-down list.<\/span><\/li>\n<li style=\"font-weight: 400;\"><span style=\"font-weight: 400;\">Select a configuration from the <\/span><b>Search Configuration<\/b><span style=\"font-weight: 400;\"> drop-down list.<\/span><\/li>\n<li>Repeat steps\u00a0 4 through 7 for multiple variants for a maximum of 4.<\/li>\n<li style=\"font-weight: 400;\"><span style=\"font-weight: 400;\">Under the <\/span><b>Traffic <\/b><span style=\"font-weight: 400;\">section, adjust the slider to set a value indicating the percent of the user query traffic that should be assigned to each variant.<\/span><\/li>\n<li style=\"font-weight: 400;\"><span style=\"font-weight: 400;\">In the <\/span><b>Duration <\/b><span style=\"font-weight: 400;\">field, enter the number of days the experiment needs to be run, up to a maximum of 90 days.<\/span><\/li>\n<li style=\"font-weight: 400;\">Click <b>Add<\/b>.<\/li>\n<\/ol>\n<\/div><\/div><div class=\"w-separator size_small with_line width_default thick_1 style_solid color_border align_center\"><div class=\"w-separator-h\"><\/div><\/div><div class=\"wpb_text_column\"><div class=\"wpb_wrapper\"><h2><span class=\"ez-toc-section\" id=\"Run_Experiments\"><\/span>Run Experiments<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<div class=\"wpb_text_column \">\n<div class=\"wpb_wrapper\">\n<p>After the addition of experiments, you can run the experiments to measure the search results based on the index and search configuration.<\/p>\n<p>To run experiments, follow the below steps:<\/p>\n<ol>\n<li>Click the\u00a0<b>Analytics\u00a0<\/b>tab on the top and select\u00a0<b>Experiments\u00a0<\/b>from the drop-down list.<\/li>\n<li>On the Experiments page, you can find the list of experiments.<\/li>\n<li>Click\u00a0<b>Run\u00a0<\/b>on the respective experiment. Note you can run only one experiment at any given point in time.<\/li>\n<li>After the experiment starts running, the status changes to\u00a0<b>Active,<\/b>\u00a0and the\u00a0<b>Duration\u00a0<\/b>is displayed.<\/li>\n<\/ol>\n<p>Note: You cannot run multiple experiments.<\/p>\n<\/div>\n<\/div>\n<\/div><\/div><div class=\"w-separator size_small with_line width_default thick_1 style_solid color_border align_center\"><div class=\"w-separator-h\"><\/div><\/div><div class=\"wpb_text_column\"><div class=\"wpb_wrapper\"><h2><span class=\"ez-toc-section\" id=\"Actions\"><\/span>Actions<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<div class=\"wpb_text_column \">\n<div class=\"wpb_wrapper\">\n<p>Once you run an experiment, you can:<\/p>\n<ol>\n<li>Monitor the progress by clicking on the experiment to view the following metrics against each variant:\n<ul>\n<li>Variant name<\/li>\n<li>Index assigned<\/li>\n<li>Search configuration<\/li>\n<li>Traffic assigned<\/li>\n<li>Number of users<\/li>\n<li>Number of searches<\/li>\n<li>Number of clicks<\/li>\n<li>Click-through rate i.e. number of unique clicks per appearances<\/li>\n<\/ul>\n<\/li>\n<li>You can choose to <strong>pause<\/strong> or <strong>stop<\/strong> the experiment.<\/li>\n<li>You can <strong>delete<\/strong> a paused or stopped experiment.<\/li>\n<li>You can <strong>edit<\/strong> the configuration of an experiment that is yet to run.<\/li>\n<\/ol>\n<\/div>\n<\/div>\n<\/div><\/div><div class=\"w-separator size_small with_line width_default thick_1 style_solid color_border align_center\"><div class=\"w-separator-h\"><\/div><\/div>[vc_raw_html]JTNDZGl2JTIwY2xhc3MlM0QlMjJ3LXBvc3QtZWxtJTIwcG9zdF9uYXZpZ2F0aW9uJTIwbGF5b3V0X3NpbXBsZSUyMGludl9mYWxzZSUyMiUzRSUwQSUwOSUzQ2ElMjBjbGFzcyUzRCUyMnBvc3RfbmF2aWdhdGlvbi1pdGVtJTIwb3JkZXJfZmlyc3QlMjB0b19wcmV2JTIyJTIwaHJlZiUzRCUyMiUyRlNlYXJjaEFzc2lzdCUyRmFuYWx5dGljcyUyRnJlc3VsdHMtaW5zaWdodHMlMkYlMjIlM0UlM0MlMkZwJTNFJTBBJTNDZGl2JTIwY2xhc3MlM0QlMjJwb3N0X25hdmlnYXRpb24taXRlbS1hcnJvdyUyMiUzRSUzQyUyRmRpdiUzRSUwQSUzQ2RpdiUyMGNsYXNzJTNEJTIycG9zdF9uYXZpZ2F0aW9uLWl0ZW0tbWV0YSUyMiUzRVByZXZpb3VzJTNDJTJGZGl2JTNFJTBBJTNDZGl2JTIwY2xhc3MlM0QlMjJwb3N0X25hdmlnYXRpb24taXRlbS10aXRsZSUyMiUzRSUzQ3NwYW4lM0VSZXN1bHRzJTIwSW5zaWdodHMlM0MlMkZzcGFuJTNFJTNDJTJGZGl2JTNFJTBBJTNDcCUzRSUzQyUyRmElM0UlM0NiciUyMCUyRiUzRSUwQSUzQ2ElMjBjbGFzcyUzRCUyMnBvc3RfbmF2aWdhdGlvbi1pdGVtJTIwb3JkZXJfc2Vjb25kJTIwdG9fbmV4dCUyMiUyMGhyZWYlM0QlMjIlMkZTZWFyY2hBc3Npc3QlMkZhcHAtbWFuYWdlbWVudCUyRm1hbmFnZS1hcHBsaWNhdGlvbiUyRiUyMiUzRSUzQyUyRnAlM0UlMEElM0NkaXYlMjBjbGFzcyUzRCUyMnBvc3RfbmF2aWdhdGlvbi1pdGVtLWFycm93JTIyJTNFJTNDJTJGZGl2JTNFJTBBJTNDZGl2JTIwY2xhc3MlM0QlMjJwb3N0X25hdmlnYXRpb24taXRlbS1tZXRhJTIyJTNFTmV4dCUzQyUyRmRpdiUzRSUwQSUzQ2RpdiUyMGNsYXNzJTNEJTIycG9zdF9uYXZpZ2F0aW9uLWl0ZW0tdGl0bGUlMjIlM0UlM0NzcGFuJTNFU2VhcmNoQXNzaXN0JTIwQXBwJTIwTWFuYWdlbWVudCUzQyUyRnNwYW4lM0UlM0MlMkZkaXYlM0UlMEElM0NwJTNFJTNDJTJGYSUzRSUzQyUyRmRpdiUzRQ==[\/vc_raw_html]<\/div><\/div><\/div><\/div><\/section>\n","protected":false},"excerpt":{"rendered":"Experiments are the best way to test multiple index and search configuration combinations and pick the best one that fulfills your business needs. SearchAssist allows you to create experiments with two or more variants to continuously improve search relevance and user experience. Overview Consider the following scenarios: Scenario 1: You have configured an index, and...","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[9],"tags":[],"jetpack_featured_media_url":"","_links":{"self":[{"href":"https:\/\/multisite.korebots.com\/SearchAssist\/wp-json\/wp\/v2\/posts\/1126"}],"collection":[{"href":"https:\/\/multisite.korebots.com\/SearchAssist\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/multisite.korebots.com\/SearchAssist\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/multisite.korebots.com\/SearchAssist\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/multisite.korebots.com\/SearchAssist\/wp-json\/wp\/v2\/comments?post=1126"}],"version-history":[{"count":8,"href":"https:\/\/multisite.korebots.com\/SearchAssist\/wp-json\/wp\/v2\/posts\/1126\/revisions"}],"predecessor-version":[{"id":1135,"href":"https:\/\/multisite.korebots.com\/SearchAssist\/wp-json\/wp\/v2\/posts\/1126\/revisions\/1135"}],"wp:attachment":[{"href":"https:\/\/multisite.korebots.com\/SearchAssist\/wp-json\/wp\/v2\/media?parent=1126"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/multisite.korebots.com\/SearchAssist\/wp-json\/wp\/v2\/categories?post=1126"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/multisite.korebots.com\/SearchAssist\/wp-json\/wp\/v2\/tags?post=1126"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}