This document provides instructions on how to set up and start a running instance of talawa-api on your local system. The instructions are written to be followed in sequence so make sure to go through each of them step by step without skipping any sections.
- Prerequisites
- Installation with Docker
- Installation without Docker
- Configuration
- Importing Sample Database
- Running Talawa-API
- How to Access the Talawa-API URL
- Accessing MongoDB
- Manually Adding The First Super Admin User
- Other
- Testing
You will need to have copies of your code on your local system. Here's how to do that.
Best way to install and manage node.js
is making use of node version managers. Two most popular node version managers right now are fnm and nvm. We'd recommend fnm
because it's written in rust
and is much faster than nvm
. Install whichever one you want and follow their guide to set up node.js
on your system.
Follow the setup guide for git
on official git docs. Basic git
knowledge is required for open source contribution so make sure you're comfortable with it. Here's a good tutorial to get started with git
and github
.
First you need a local copy of talawa-api. Run the following command in the directory of choice on your local system.
- Navigate to the folder where you want to setup the repository. Here, I will set it up in a folder called
talawa
. - Navigate to the folder and open a terminal in this folder (you can right-click and choose appropiate option based onn your OS). Next, we'll fork and clone the
talawa-api
repository. - Navigate to https://github.com/PalisadoesFoundation/talawa-api/ and click on the
fork
button. It is placed on the right corner opposite the repository namePalisadoesFoundation/talawa-api
. - You should now see
talawa-api
under your repositories. It will be marked as forked fromPalisadoesFoundation/talawa-api
- Clone the repository to your local computer (replacing the values in
{{}}
):
$ git clone https://github.com/{{YOUR GITHUB USERNAME}}/talawa-api.git
This will setup the repository and the code files locally for you. For more detailed instructions on contributing code, and managing the versions of this repository with Git, checkout CONTRIBUTING.md here
NOTE: All the commands we're going to execute in the following instructions will assume you are in the root directory of the project. If you fail to do so, the commands will not work.
Install the packages required by talawa-api
using this command:
npm install
It's important to configure Talawa-API to complete it's setup. You can use our interactive setup script for the configuration. Use the following command for the same.
npm run setup
All the options in "setup" can be done manually as well and here's how to do it. - The .env Configuration File
- Requires Docker and Docker Compose to be installed
- Will start a local mongodb and redis instances
Now use the following command to run docker containers -
docker compose up
OR
docker-compose up
Note: If you're using Docker, you'll need to manually import the sample data after the Docker Compose has started the MongoDB container. For instructions on how to do this, refer to Importing Sample Database
Talawa-api makes use of MongoDB
for its database needs. We make use of mongoose ODM
to interact with the MongoDB database from within the code.
We're listing some common approaches to set up a running instance of MongoDB database:
System native database approach:
(Highly Recommended) You can install MongoDB natively on your system and create/connect to the database. Follow the setup guide on official MongoDB Docs for your respective operating system.Hosted database approach:
MongoDB Atlas is the easiest way to get a running instance of mongodb database. It is a hosted(remote) mongodb database provided by mongodb itself. If you're a beginner and don't want too much of a hassle setting up the database you should use this approach but you should eventually switch to local instance. Follow the setup guide on official MongoDB Atlas Docs. Mongodb Atlas is just one of the many hosted database solutions. Some issues that you might face while using this are slower tests, slower API requests, dependence on Internet connection etc.Docker container approach:
If you are fluent in working with docker you should use this approach. Docker is a great way to manage and run applications without natively installing anything on your system. With this you can set up the mongodb database inside a docker container and manage it as per your will. Follow this video tutorial to set up a mongodb docker container. You can learn about docker from Docker docs.
Talawa-api makes use of Redis
for caching frequently accessed data items in the primary database. We make use of ioredis
to interact with the redis-server
from within the code. The main Idea is the in production this will act as an in-memory cache. So it is recommended that you set it up locally. However for simplicity purposes, a section to accomodate for setting Redis up using a remote instance like Redis Cloud has been added. Please note that this is not recommended since the remote connection takes a considerable amount of time to be considered as a cache to improve application performance.
-
For Linux Users
:If you are using a Linux distribution, follow these steps to set up Redis:
-
Step 1: Open a terminal.
-
Step 2: Update the package list:
sudo apt update
-
Step 3: Install Redis Server:
sudo apt install redis-server
-
Step 4: Start the Redis service:
sudo service redis-server start
-
Step 5: Test if Redis is running by running the Redis CLI:
redis-cli
-
-
For Windows Users using WSL
:
If you'd rather not deal with the hassle of setting up WSL on your computer, there's another option: you can use a hosted database like Redis Cloud. More details about this are provided below, mainly for when you're working on development tasks. But it's a good idea to set up Redis on your own computer if you can. Right now, Redis isn't supported directly on Windows – you can only install and use it through WSL. If you're a Windows user and want to get Redis working using the Windows Subsystem for Linux (WSL), just follow these steps:
-
Step 1: Install WSL (Windows Subsystem for Linux) following the official WSL Installation Guide.
-
Step 2: Open a WSL terminal.
-
Step 3: Update the package list:
sudo apt update
-
Step 4: Install Redis Server:
sudo apt install redis-server
-
Step 5: Start the Redis service:
sudo service redis-server start
-
Step 6: Test if Redis is running by running the Redis CLI:
redis-cli
-
Connecting to Redis Cloud:
To connect to a Redis cloud service, you will need the host and port information provided by your cloud service provider. Use these values in your application to establish a connection. Typically, the host and port strings are provided in the following format:
- Host:
your-redis-host.redisprovider.com
- Port:
6379
(default Redis port)
Replace
your-redis-host.redisprovider.com
with the actual host provided by your Redis cloud service. You can then use these values in your application's configuration to connect to your Redis cloud instance. You may also have to enter Redis Password and Username for using cloud instance. - Host:
Remember to adjust any paths or details as needed for your specific environment. After following these steps, you will have successfully set up Redis.
It's important to configure Talawa-API to complete it's setup.
You can use our interactive setup script for the configuration. Use the following command for the same.
npm run setup
It can be done manually as well and here's how to do it.
A file named .env
is required in the root directory of talawa-api for storing environment variables used at runtime. It is not a part of the repo and you will have to create it. For a sample of .env
file there is a file named .env.sample
in the root directory. Create a new .env
file by copying the contents of the .env.sample
into .env
file.
cp .env.sample .env
This .env
file must be populated with the following environment variables for talawa-api to work:
Variable | Description |
---|---|
NODE_ENV | Used for providing the environment in which the |
the talawa-api is running | |
ACCESS_TOKEN_SECRET | Used for signing/verifying JWT tokens |
REFRESH_TOKEN_SECRET | Used for signing/verifying JWT tokens |
MONGO_DB_URL | Used for connecting talawa-api to the mongoDB database |
RECAPTCHA_SECRET_KEY | Used for authentication using reCAPTCHA |
RECAPTCHA_SITE_KEY | Used for authentication using reCAPTCHA |
MAIL_USERNAME | Used for mailing service |
MAIL_PASSWORD | Used for mailing service |
LAST_RESORT_SUPERADMIN_EMAIL | Used for promoting the default super admin |
COLORIZE_LOGS | Used for colorized log formats in console |
LOG_LEVEL | Used for setting the logging level |
REDIS HOST | Used for connecting talawa-api to the redis instance |
REDIS_PORT | Specifies the port of the active redis-server |
REDIS_PASSWORD(optional) | Used for authenticating the connection request to |
a hosted redis-server |
The following sections will show you how to configure each of these parameters.
Change the environment from production to development:
NODE_ENV=development
Access and refresh token secrets are used for authentication purposes.
Run the following command and copy/paste the result to the variable named ACCESS_TOKEN_SECRET
in .env
file.
The command to use is:
openssl rand -hex 32
This command is available if you have Git for Windows installed. Follow these steps:
- Install
Git for Windows
- Run the Git Bash app
- Issue the same command
openssl rand -hex 32
Run the following command and copy/paste the result to the variable named REFRESH_TOKEN_SECRET
in .env
file.
openssl rand -hex 32
Here's how you will configure MongoDB.
NOTE: Talawa-API uses 2 databases, a primary and test version.
- You only have to setup one database and provide it's URL in the
.env
file. This is theprimary database
and is used to store all your data. - We automatically create a new database with the name
TALAWA_API_TEST_DATABASE
. This is exclusively used for storing all the test data generated during the testing process so that it does not bloat the main database with unnecessary data.
A Connection String
is the URL that applications use to access a MongoDB database. Talawa-API will need to know the correct connection string to use to perform correctly.
- The
Connection String
is the.env
variable namedMONGO_DB_URL
in the.env
file. - The
Connection String
can differ depending on the approach you used to set up your database instance. Please read the official mongodb docs onconnection string
.
Your MongoDB installation may include either the mongo
or mongosh
command line utility. An easy way of determining the connection string
is to:
- Run the command line utility
- Note the
connection string
in the first lines of the output. - Add the first section of the
connection string
to theMONGO_DB_URL
section of the.env
file. In this case it ismongodb://127.0.0.1:27017/
$ mongosh
Current Mongosh Log ID: e6ab4232a963d456920b3736
Connecting to: mongodb://127.0.0.1:27017/?directConnection=true&serverSelectionTimeoutMS=2000&appName=mongosh+1.6.2
Using MongoDB: 6.0.4
Using Mongosh: 1.6.2
For mongosh info see: https://docs.mongodb.com/mongodb-shell/
...
...
...
...
There are a few more steps that need to be done in a Windows environment.
- Download the MongoDB Shell from the tools section at the following link:Mongo Shell
- Extract the downloaded shell folder, locate the
mongosh
application, and paste it to the following location:Program Files
->MongoDB
->bin
.- You will find the mongosh application inside the
bin
folder]
- You will find the mongosh application inside the
- Add the path of the location where you pasted the
mongosh
application to your system's environment variables. - In a separate terminal, run the
mongod
command to start the local instance of the database. - Create a folder named "data" in the C drive and within it create a new folder named "db".
- Open a terminal and run the
mongosh
command in the terminal you will get the connection string. In this case the Connection String is:mongodb://127.0.0.1:27017/?directConnection=true&serverSelectionTimeoutMS=2000&appName=mongosh+1.6.2
- In the
.env
file of Talawa-API, add the connection string to theMONGO_DB_URL
section.
- In the
$ mongosh
Current Mongosh Log ID: e6ab4232a963d456920b3736
Connecting to: mongodb://127.0.0.1:27017/?directConnection=true&serverSelectionTimeoutMS=2000&appName=mongosh+1.6.2
Using MongoDB: 6.0.4
Using Mongosh: 1.6.2
For mongosh info see: https://docs.mongodb.com/mongodb-shell/
...
...
...
...
Here's the procedure to set up Redis.
In the .env
file, you should find three variables: REDIS_HOST
, REDIS_PORT
, and REDIS_PASSWORD
. These environment variables will contain the necessary information for your codebase to connect to a running redis-server
.
In both scenarios (Linux or WSL post-installation), the variable values should be configured as follows:
REDIS_HOST
= localhostREDIS_PORT
= 6379 Note: This default port is used by theredis-server
. However, if yourredis-server
operates on a different port, you must provide that port number.REDIS_PASSWORD
should be left empty, as passwords are unnecessary for local connections.
To begin, you must register for a free account on Redis Cloud. Following this step, you can proceed by selecting a database from the free tier, which comes with a 30MB data storage limit. Once completed, you can then access your Database by navigating to the Databases
section. Here, you will find the option to view the overall settings of your free instance.
Here are the configuration details:
REDIS_HOST
= ThePublic endpoint
assigned to your Database, excluding the.com
. It will resemble something likeredis-13354.c169.us-east-1-1.ec2.cloud.redislabs.com
. The numerical value following this address is the port number.REDIS_PORT
= The number provided in thePublic Endpoint
after the colon (:
), for instance:13354
.REDIS_PASSWORD
= TheDefault user password
located in the Security Section.
The user with the email address set with this parameter will automatically be elevated to Super Admin status on registration.
- When installing, set this to the email address of the person you want to be the very first Super Admin.
- This will usually be the email address of the person installing the software.
- If this is not set you will not be able to administer the application.
If you don't set this parameter, then you'll need to follow the Manually Adding The First Super Admin User
process discussed later in this document.
Set this value in the event that you need to elevate any of your users to be a Super Admin.
NOTE It is STRONGLY advised that you remove the email address once the initial installation and setup has been done.
You need to have a google
account to follow the following steps.
We use reCAPTCHA
for two factor authentication (2FA). Follow these steps:
-
Visit the reCAPTCHA Key Generation URL.
-
Click on
Submit
button. -
Copy the generated
Secret Key
to variable namedRECAPTCHA_SECRET_KEY
in.env
file. -
NOTE: Save the generated
Site key
as it will be used intalawa-admin
.
NOTE: ReCAPTCHA is a type of 2FA, so your Google account needs to have two factor authentication set up for the following steps to work. Make sure this is done before proceeding
The MAIL_USERNAME and MAIL_PASSWORD parameters are required to enable an app to access 2FA features. This is how to know what they should be.
- Go to your google account page.
- Select
Security
. - Under
Signing in to Google
section selectApp Passwords
. - Click on
Select app
section and chooseOther(Custom name)
, entertalawa
as the custom name and pressGenerate
button. - Copy the 16 character generated app password to the variable named
MAIL_PASSWORD
in.env
file. - Copy your usual gmail address to the variable named
MAIL_USERNAME
in.env
file.
For more info refer to this Google Answer.
For using SMTP server instead of Gmail, following steps need to be followed:
- Set the
IS_SMTP
variable totrue
for exampleIS_SMTP=true
- Go to your your SMTP server, and note the following variables:
SMTP_HOST=your-smtp-server-hostname
SMTP_PORT=your-smtp-server-port
SMTP_USERNAME=your-smtp-username
SMTP_PASSWORD=your-smtp-password
SMTP_SSL_TLS=true-or-false
For example:
SMTP_HOST=smtp.hostgator.com
SMTP_PORT=465
[email protected]
SMTP_PASSWORD=
SMTP_SSL_TLS=true
For more information on setting up a smtp server, here's a useful article
You can set up and customize logs by configuring the following parameters
The parameter COLORIZE_LOGS
is a boolean field and can be set to true or false. It customizes the log colorization formats displayed in console. You can set the value in .env
file as
COLORIZE_LOGS = false
If the parameter value is set to true
, you should be able to see colorized logs in console, or else logs will display in the console's default simple format.
There are different logging levels that can be configured by setting this parameter. The severity order of levels are displayed numerically ascending from most important to least important.
levels = {
error: 0,
warn: 1,
info: 2,
http: 3,
verbose: 4,
debug: 5,
silly: 6
}
On setting this parameter value, log messages are displayed in the console only if the message.level
is less than or equal to setted LOG_LEVEL
For our application, the most appropriate setting is LOG_LEVEL = info
since most of information logged on the console are error messages, warnings or info texts.
You need to have a google
account to follow the following steps.
We use firebase for mobile app notifications. To configure the notification service create a new firebase project and follow these steps:-
- Create a new Firebase project for Talawa-API
- When created you will automatically enter the project's console area
- Click on the settings icon beside the
Project Overview
heading - Click on
Project Settings
- Click on the
Service Accounts
tab - Click on the
Node.js
radio button - Click on
Generate New Private Key
button - Confirm by clicking on
Generate Key
. This will automatically download the private keys in your browser. - Securely store the
JSON
file containing the private key. These will be used in the next section.
The key generated in the previous step is in a format suitable for use in a mobile app. We need to convert it for use by the API. This will require you to do some work in the talawa repository to do the necessary conversion. The resulting output will be stored in a lib/firebase_options.dart
file. Some of the contents of this file will then need to be added to the API's .env
file. Here we go.
-
Clone the talawa mobile app in a separate directory that is not under your Talawa-API directory.
-
Enter that directory as you will need to edit files there
-
Run the following commands to set the key in the environment variable for your respective operating system:
-
Linux/macOS:
export GOOGLE_APPLICATION_CREDENTIALS="/PATH/TO/JSON/FILE/filename.json"
-
Windows:
$env:GOOGLE_APPLICATION_CREDENTIALS="C:\PATH\TO\JSON\FILE\filename.json"
-
-
Install the Firebase CLI.
-
Save the original copy the
lib/firebase_options.dart
file as it will be modified. -
Run the following commands in the project directory of talawa mobile app:
firebase login dart pub global activate flutterfire_cli
-
Run any commands about exporting variables from the previous
dart
command. -
Run the following command to configure the application for Firebase
flutterfire configure
-
Select the project you created in the firebase console.
-
Add
iOS
andandroid
platforms to the project. -
Overwrite the
firebase_options.dart
file if asked so. -
The command will generate keys for the
iOS
andandroid
platforms respectively and place them in thefirebase_options.dart
file. -
Edit the
firebase_options.dart
file. -
Add the parameters in the
static const FirebaseOptions android = FirebaseOptions
section of thefirebase_options.dart
file to the Talawa API.env
file under theandroidFirebaseOptions
heading.-
Replace any parameters that are already there in that section.
-
Remove any trailing commas on the lines you have added.
-
Remove any leading spaces on the lines you have added.
-
The final result in the
.env
file should look like thisapiKey: '9f6297b283db701dab7766c993c48b', appId: '1:261699118608:android:366ff7dbdfba5c5a9e8392', messagingSenderId: '261699118608', projectId: 'talawa-thingy', storageBucket: 'talawa-thingy.appspot.com',
-
-
Add the parameters in the
static const FirebaseOptions ios = FirebaseOptions
section of thefirebase_options.dart
file to the Talawa API.env
file under theiosFirebaseOptions
heading. Replace any paramters that are already there.-
Replace any parameters that are already there in that section.
-
Remove any trailing commas on the lines you have added.
-
Remove any leading spaces on the lines you have added.
-
The final result in the
.env
file should look like thisapiKey: 'c2d283aa45f4e858c9cbfe32c58c67', appId: '1:261699118608:ios:1babbb3c07b8461ebdcb2', messagingSenderId: '261699118608', projectId: 'talawa-thingy', storageBucket: 'talawa-thingy.appspot.com', iosClientId: '261699118608-d519b739e43c6214374c0da62feaef.apps.googleusercontent.com', iosBundleId: 'com.example.talawa',
-
-
Undo the changes made to the
firebase_options.dart
file by overwriting it with the version you saved at the beginning of this section.
Talawa API contains a sample database importing function which can be used to import sample database.
npm run import:sample-data -- [args]
You can pass the following arguments while running this script.
--format
: Cleans the database before import. Add this flag with caution. It will delete all of the existing data inside the talawa database.--items=
: Specify the items to add.- Following
items
can be specified, separated with a comma,
users
: For users collectionorganizations
: For organizations collectionevents
: For events collectionposts
: For posts collection
- Following
npm run import:sample-data
: This command will import the complete sample database without removing the existing data.npm run import:sample-data -- --format
: This command will import the complete sample database after removing the existing data.npm run import:sample-data -- --format --items=users,organizations
: This command will import the sampleusers
andorganizations
collections after cleaning the existing data.npm run import:sample-data -- --items=users,organizations
: This command will import the sampleusers
andorganizations
collections without cleaning the existing data.
The sample data contains organizations, users, events and posts. Here are the details for each of organizations and user account.
Password | User Type | Joined Organization | Admin For | |
---|---|---|---|---|
[email protected] | Pass@123 | USER | Angel Foundation | None |
[email protected] | Pass@123 | USER | Angel Foundation | None |
[email protected] | Pass@123 | USER | Angel Foundation | None |
[email protected] | Pass@123 | ADMIN | Angel Foundation | Angel Foundation |
[email protected] | Pass@123 | ADMIN | Hope Foundation | Hope Foundation |
[email protected] | Pass@123 | ADMIN | Dignity Foundation | Dignity Foundation |
[email protected] | Pass@123 | SUPERADMIN | The Unity Foundation | ALL |
There is one post inside the The Unity Foundation
There is one event inside the The Unity Foundation
Talawa-api development server runs two processes simultaneously in the background. They are:
-
GraphQL code generator:
This watches for changes in the graphQL type definition files and generates corresponding typescript types in the background. This results in good code editor experience with typescript. -
Talawa-api server:
This runs talawa-api directly transpiling the typescript files and running them without emitting as javascript files. It also watches for changes in the code files and restarts the server if it detects any changes.
Run the following command to start talawa-api development server:
npm run dev
There are many important URLs for accessing the API
By default talawa-api runs on port 4000
on your system's localhost. It is available on the following endpoint:
http://localhost:4000/
If you navigate to the endpoint you and see a JSON
response like this it means talawa-api is running successfully:
{"talawa-version":"v1","status":"healthy"}
GraphQL endpoint for handling queries
and mutations
is this:
http://localhost:4000/graphql/
GraphQL endpoint for handling subscriptions
is this:
ws://localhost:4000/graphql/
The Organization URL for Talawa mobile app developers will depend upon the device on which Mobile app is installed.
- If Talawa Mobile App is installed on Android Virtual Device (AVD), use the following URL:
http://10.0.2.2:4000/graphql
- If Talawa Mobile App is installed on a Real Mobile Device, follow the below steps to get URL:
- Open Command Prompt in Windows, or Terminal in Linux/OSX
- Enter
ipconfig
(For Windows Users) orifconfig
(For Linux/OSX Users) - Your Mobile and Computer (On which API server is running) must be on same Wifi Network. Use Mobile Hotspot to connect your computer to internet in case you don't have access to a Wifi Router.
- Search for the
Wireless LAN adapter Wi-Fi:
and then copyIPv4 Address
, like in image below: - Now, use this IP address (
192.168.0.105
in our case) to access the API instance using the following URL pattern:
http://{IP_Address}:4000/graphql
For example:
http://192.168.0.105:4000/graphql
The Organization URL for Talawa mobile app developers to use is:
http://localhost:4000/graphql/
There are many ways to access MongoDB.
Open MongoDB Compass and click on "Connect." You will now be able to access the graphical user interface of the local database.
NOTE: You can do the same in macOS and linux with minor tweaks. This has been provided to give a brief overview for beginners to setup their own local instance.
This guide is for VSCode
users to easily manage their MongoDB
databases:
-
Install the offical
MongoDB
extension forVSCode
namedMongoDB for VS Code
. -
Connect your
MongoDB
database to the extension. -
Now you can manage the database you are using for
talawa-api
through this extension withinVSCode
.
You can skip these instructions for now if you don't have running instance of Talawa-Admin.
-
This step is for mandatory Linux specific users others can skip to next step:
- You need to start
mongod
[Mongo daemon process] formongosh
to work use the following command for the same:
sudo service mongod start
[System V init(service)] orsudo systemctl start mongod
[systemd(systemctl)]
- To verify whether
mongod
[Mongo daemon process] is running you can use either:
sudo systemctl status mongod
[systemd(systemctl)] orsudo service mongod status
[System V init(service)]
- You need to start
-
Open MongoDB Compass and click on
Connect
. -
Select
user
collections and edit the data. Change:userType
fromUSER
toSUPERADMIN
adminApproved
fromfalse
totrue
-
Open a terminal and run
mongosh
command to open interactive command line interface to work with MongoDB database. -
In the
mongosh
terminal use the following command to edit theusers
collections data: 1.Find the login credentials in the database through following command:db.users.find({userType: 'USER', firstName: '<user's first name>'})
- Elevate permission from
USER
toSUPERADMIN
and setadminApproved
totrue
:
db.users.updateOne({ firstName: '<user's first name>' },{ $set: { userType: 'SUPERADMIN', adminApproved: true }})
- To verify the details were updated correctly use:
db.users.find({firstName:'<user's first name>' })
- Elevate permission from
Note: You can do the edits via any of the two methods.
These are some other factors to consider
If port 4000
is not free on your system you can pass a custom environment variable named PORT
to the script to make it use a different port on your system's localhost. Here's the syntax for it:
PORT=<CUSTOM_PORT_VALUE> npm run dev
where <CUSTOM_PORT_VALUE>
is whatever value you want the PORT
to be. Whatever you pass will be substituted as the value for port and talawa-api development server on that port. Syntax wise it looks like-
http://localhost:<CUSTOM_PORT_VALUE>/
For example:
PORT=5000 npm run dev
will make talawa-api accessible on the following endpoint:
http://localhost:5000/
Talawa-api makes use of vitest
to run tests because it is much faster than jest
and more comfortable to work with.
You can run the tests for talawa-api using this command:
npm run test