# Migration from Database Full-text Search to Elasticsearch Index

To use this plugin, all elements needs to be indexed again into the Elasticsearch database. Depending on the number of element, number of sites and the size of the underlying infrastructure, this process cann take quite a long time. To make this transition as smooth as possible, our plugin offers a few dedicated migation features.

# Migration steps

# Executive Summary

Install and setup plugin
Turn Transition Mode on
Re-index all elements
Grab a coffee or go on vacation (depending on how many elements need to be re-indexed)
Turn Transition Mode off
Truncate old searchindex table
Work like before, but with faster search results

# In-depth Guide

After installation, the settings for the plugin must be configured. As long as the setting for the endpoint is empty or the environment variable used returns an empty string, the plugin is not active and does not make any changes to the full text search process. Only when a valid endpoint is configured, the plugin becomes active. Therefore, it can be installed without affecting the system.
The Transition mode is turned on after the installation by default. This ensures, that during the potentially time-consuming process of re-indexing, all elements that are already stored in the Elasticsearch index, but have been updated in the meantime, are also kept up to date. This is done by updating both, the database full-text index and the Elasticsearch index at the same time, keeping them consistent.

If the search is used by a user during Transition mode is on, only the database full-text index is used.
Now the re-indexing of all elements must be done. There are two ways to achieve this:

TIP

This part of the migration should be planned and tested on a test environment. This will give you a better understanding of whether the path makes sense and how long it will take to index all the elements.

Use Craft's own function of re-saving elements with the option to rebuild the search index:
```
./craft resave/entries --update-search-index
```
Cons: This command must be executed for all elements (Assets, Categories, Entries, Matrix Blocks, Tags and Users) separately. This process runs in the foreground, can take a long time and must not be interrupted. Otherwise you start from the beginning. Another drawback: It does not include elements that have been registered by other plugins and should also be added to the full-text search.
The plugin comes with a console command that creates a job in the queue for each element, so that even in case of an interruption the process can be continued where it stopped. Furthermore, the command includes all element types registered by plugins:
```
./craft elastic/elements/index
```
We recommend using the second option. However, it should be kept in mind that really for every element a job is created in the queue. This can lead to performance degradation if there are a large number of elements in Craft. Moreover, the queue should not be triggered by the frontend, but processed by a separate worker. However, most users considering using this plugin, will certainly already have such an environent.

We further recommend setting up a separate queue only for these migration jobs and having them processed by one (or more) workers. If you want to run multiple workers without losing speed (due to locking), we recommend using a different technology for the queue like AWS SQS, instead of the default Craft database.

The console command allows to specify a different queue and to change the priority of the jobs (if supported by the queue). By default, the default queue and priority 2048 is used. This ensures that when the default queue is used, all important jobs of Craft are given priority (they all have 1024 as priority). For example:
```
./craft elastic/elements/index migrationQueue 4096
```

After all elements have been added to the Elasticsearch index, the Transition mode can be turned off. Please be aware that from this point all updates will be made only in Elasticsearch. Therefore, you should make sure that the search is working properly and avoid making any changes to elements during the test.

TIP

It is possible to switch back to the transition mode at any time and thus have the search served by the database full text search again. The plugin remembers the time when the transition mode was deactivated and, after returning to this mode, adds all elements created or updated since that point in time to the outdated database search, so that it is up to date again.

If everything works to your satisfaction, you can truncate the database table of the old index. It is no longer used and will reduce the size of the database significantly.
```
$ ./craft elastic/migration/truncate-table
```

← Installation & setup Console Command →