🌎 Populate your database using the GeoNames service

🗒️ Description

The package is useful for applications that rely on the geo data.

By default, it provides 4 models: Continent, Country, Division, City and translations for them.

The translations are powered by the nevadskiy/laravel-translatable package.

The package also keeps the data up-to-date by fetching daily modifications provided by the GeoNames service and uses them to synchronize your own database.

🔌 Installation

composer require nevadskiy/laravel-geonames

If you are going to use translations, you also need to install an additional package.

composer require nevadskiy/laravel-translatable

✅ Requirements

• Laravel 8.0 or newer
• PHP 7.3 or newer

🔨 Usage

Publish the package resources using the command:

php artisan vendor:publish --tag=geonames-migrations --tag=geonames-models

Seeding

Before seeding, make sure you run the database migrations.

php artisan migrate

Then, run the seeding process.

php artisan geonames:seed

It will download geonames resources and insert the dataset into your database.

Note that the seeding process may take some time. On average, it takes about 40 minutes (without downloading time) to seed the full dataset with translations.

If you have issues with memory leaks during seeding, check out this section.

Schedule updates

Add the following code to the app/Console/Kernel.php file if you want to receive geonames daily updates.

Geonames daily updates are published at 3:00 in the UTC time zone, so to be sure that they are already available, it is recommended to run the command a bit later.

protected function schedule(Schedule $schedule)
{
    $schedule->command('geonames:daily-update')->dailyAt('4:00');
}

Note that time is specified for the UTC timezone.

Syncing

If you missed some daily updates or just decided to change seeder filters, you can sync your database records according to the current geonames dataset.

php artisan geonames:sync

This command will create missing records, remove redundant ones, and updated modified ones according to the current dataset.

Note that the geoname_id and alternate_name_id fields is required to synchronize data.

Customization

If you want to customize migrations or data that should be imported, you can simply do this by overriding the default seeders.

To do that, publish the package seeders using command:

php artisan vendor:publish --tag=geonames-seeders

Publish the package config:

php artisan vendor:publish --tag=geonames-config 

Then, specify published seeders in the config/geonames.php file:

'seeders' => [
    Database\Seeders\Geo\ContinentSeeder::class,
    Database\Seeders\Geo\ContinentTranslationSeeder::class,
    Database\Seeders\Geo\CountrySeeder::class,
    Database\Seeders\Geo\CountryTranslationSeeder::class,
    Database\Seeders\Geo\DivisionSeeder::class,
    Database\Seeders\Geo\DivisionTranslationSeeder::class,
    Database\Seeders\Geo\CitySeeder::class,
    Database\Seeders\Geo\CityTranslationSeeder::class,
]

Filtering

To reduce the database size, you can set up filters for seeding only those geo data that you really need in your application.

For example, you can set the minimum population for the city. All cities with smaller population will not be imported.

To do that, override the $minPopulation property in the CitySeeder class.

To have full control over this behaviour, override the filter method of the seeder.

Attributes mapping

To add custom fields to the table, you also need to tell the seeder how to fill those fields using the mapAttributes method.

The mapAttributes method should return all attributes of the database record, including timestamps, because model events will not be fired during seeding process since the package uses a bulk insert strategy. However, all model casts and mutators will be applied as usual.

For example, if you want to use UUIDs as primary keys, you can extend the original seeder as following:

<?php

namespace Database\Seeders\Geo;

use App\Models\Geo\City;
use Illuminate\Support\Str;use Nevadskiy\Geonames\Seeders\CitySeeder as Seeder;

class CitySeeder extends Seeder
{
    /**
     * {@inheritdoc}
     */
    protected static $model = City::class;

    /**
     * {@inheritdoc}
     */
    protected function mapAttributes(array $record): array
    {
        return array_merge(parent::mapAttributes($record), [
            'id' => (string) Str::uuid(),
        ]);
    }
}

Custom seeders

For a more significant change in the structure, you can add your own seeders or extend existing ones.

Each seeder must implement the Nevadskiy\Geonames\Seeders\Seeder interface.

All seeders that are specified in the geonames config file will be executed one by one in the specified order.

Translations

To use translations you need to install the nevadskiy/laravel-translatable package.

Read its documentation to learn more about how it works. You can also use it to handle translations of other models.

Otherwise, you still can use the package without translations, just simply remove the following:

• translation migrations
• translation seeders (from the geonames config file as well)
• the HasTranslations trait and translatable prop from published models

Memory leaks

One of the most popular issues associated with seeding large amounts of data is a memory leak.

This package reads files using PHP generators and lazy collections to avoid loading the entire file into memory.

However, there are packages that log database queries and model events during long-running commands to memory, which leads to memory leaks.

There are instructions on how to avoid memory leaks when working with the following packages:

Laravel Ignition

Publish flare config using php artisan vendor:publish --tag=flare-config.

Set report_query_bindings to false in the flare.php config file as following:

'flare_middleware' => [
    ...
    AddQueries::class => [
        'maximum_number_of_collected_queries' => 200,
        'report_query_bindings' => false,
    ],
    ...
]

Laravel Telescope

Update the telescope.php config file as following:

'ignore_commands' => [
    'geonames:seed',
    'geonames:daily-update',
    'geonames:sync',
]

📑 Changelog

Please see CHANGELOG for more information what has changed recently.

☕ Contributing

Please see CONTRIBUTING for more information.

📜 License

The MIT License (MIT). Please see LICENSE for more information.