Setting up an ES for development

This pages will get you started with developing on the ES. It is quit easy to get started as a developer because the ES can be run as virtual machine with all software installed, so all you need to do to get start developing is to setup the development environment by adding the development plugin, then checkout the source code with git.

For working on the source code one normally create one’s own private Git repository, known as a "fork" and then send changes back to Searchdaimon. Some trusted developers also have direct access to the Searchdaimon Git repository.

Table of Contents

1. Getting an ES up and running
2. Setting up development environment and ssh login
3. Seting up SSH communication to GitHub
4. Geting the code into the ES
5. Compiling the source code
6. Crawling some test data
7. Tips & tricks

Getting an ES up and running

Start by obtain a running ES. For example by downloading an ES virtual machine image from http://www.searchdaimon.com/download/ or use one of the torrents in the torrent folder. Import and run on VMware/Xen/VirtualBox. Alternatively you can run a version on Amazon Web Services or install from an ISO image.

Development environment and SSH login

1. Access your ES thru the console and login as root (the password will be visible in the console). Use yum to setup the development environment and dependencies:
   yum install boitho-devel

2. Disable automatic updates. You must disable automatic updates to prevent that new official ES updates overwrites your changes. Open the searchdaimon.repo file for editing:
   nano -w /etc/yum.repos.d/searchdaimon.repo
   Find the [boitho-production] section and set it from “enabled=1” to “enabled=0”.

3. Run the enable-root-login.sh to allow for ssh logins (this step is not necessary on Amazon Web Services):
   sh /home/boitho/boithoTools/script/enable-root-login.sh

4. Change the root password (this step is not necessary on Amazon Web Services):
   passwd

5. Use a ssh client to log on to the ES as root. Then become the boitho user:
   su – boitho
   Go to the ES home directory.
   cd boithoTools

#Seting up SSH communication to GitHub If you are ever going to submit code back to a GitHub repository, Searchdaimons or your own fork, you will have to setup git and a ssh key so you can authenticate yourself to GitHub.

Generate a new SSH key for GitHub communication

To generate a new SSH key you have to be logged in as the boitho user, then enter the code below. Replace [email protected] with the email address you are using in GitHub. We want the default settings so when asked to enter a file in which to save the key, just press enter.

ssh-keygen -t rsa -C "[email protected]"
# Creates a new ssh key, using the provided email as a label
# Generating public/private rsa key pair.
# Enter file in which to save the key (/c/Users/you/.ssh/id_rsa): [Press enter]

Adding a passphrase is optional.

Display your newly generated public key:

cat ~/.ssh/id_rsa.pub

Add your SSH key to GitHub

1. At GitHub.com go to your Account Settings
2. Click "SSH Keys" in the left sidebar
3. Click "Add SSH key"
4. Copy & paste your key into the "Key" field
5. Click "Add key"
6. Confirm the action by entering your GitHub password Run the following code to show the key.

( freely transformed from https://help.github.com/articles/generating-ssh-keys )

Geting the code into the ES

Method 1: Forking the Searchdaimon repository

Start by going to https://github.com/searchdaimon/enterprise-search and press the Fork button to make your own private copy of the repository.

Setup on the ES

You will then have to setup your newly forked repository on the ES.

Start by setting up your name and email. Replace your name and email address with your own.

git config --global user.name "Your name"
git config --global user.email "[email protected]"

Setup Git to use your forked repository. Replace [email protected]:you/enterprise-search.git with the correct path to your repository.

git clone --no-checkout [email protected]:you/enterprise-search.git /home/boitho/boithoTools.git

Setup the .git repository by running:

mv /home/boitho/boithoTools.git/.git /home/boitho/boithoTools/
cd /home/boitho/boithoTools/
git reset --hard HEAD
git pull

Adding the main repository as upstream

Add the main Searchdaimon repository as a source alos. That way you can easy get updates from the main Searchdaimon repository, but your changes gos to your own repository.

git remote add upstream [email protected]:searchdaimon/enterprise-search.git

Then in the further you may run this to merge changes from the main Searchdaimon repository into yours:

git fetch upstream
git merge upstream/master

Optional

You may want to set the editor for writing commit messages to something more user friendlier than Vim if you are not familiar with it. For example nano:

git config --global core.editor "nano"

Compiling the source code

Compile all the ES source code and restart the daemons by running:
make

More information is available at: Compiling the source code

Method 2: You have direct access

If you are a member of the Searchdaimon core team you will have direct access.

Start by setting up your name and email. Replace your name and email address with your own.

git config --global user.name "Your name"
git config --global user.email "[email protected]"

Setup Git and import the code from GitHub:

git clone --no-checkout [email protected]:searchdaimon/enterprise-search.git /home/boitho/boithoTools.git
mv /home/boitho/boithoTools.git/.git /home/boitho/boithoTools/
cd /home/boitho/boithoTools/
git reset --hard HEAD
git pull

Optional

You may want to set the editor for writing commit messages to something more user friendlier than Vim if you are not familiar with it. For example nano:

git config --global core.editor "nano"

Compiling the source code

Compile all the ES source code and restart the daemons by running:
make

More information is available at: Compiling the source code

Tips & tricks

Firewall

yum uses http and git is tunneled over ssh, so remember to open ports 80 and 22 if you have an external firewall.