This issue outlines a plan to replace redirects from .htaccess configuration files in the ansible/docsite repository with stub pages.

The purpose of this plan is to facilitate the migration of Ansible community documentation from Red Hat managed infrastructure to Read The Docs hosting. Redirects are created when pages move to avoid 404 errors. However we have more redirects now than we can add to projects on Read The Docs.

Stub pages

Stub pages are RST files that use the orphan metadata field so that the page is generated at build time but not included in the toctree. The result is that the stub page does not appear in the navigation or documentation structure, while still being reachable from a direct external link such as a bookmark or reference in a third-party site.

Stub pages provide an alternative to dynamic HTTP redirects that prevent 404 errors and broken links. They have the following benefits:

• Avoid degradation of SEO authority compared with 404s.
• Allow community contributors to restructure documentation when access to server-side redirects is not available.
• Encourage users to update stale bookmarks and external links. This is arguably a disadvantage because it requires manual intervention.
• Easier to maintain than redirects because there is no need for regex or special knowledge.

Stub pages also have some drawbacks:

• Dynamic redirects are more efficient for users because they do not require manual intervention to access pages that have been moved or removed.
• Stub pages can add clutter to the source repository. A large number of unused files can be confusing for new contributors and more difficult to navigate. Although we can put most stubs in a special folder to avoid polluting active directories.
• Adds to the build time. While stub pages are not included in the toctree, they still need to be generated into HTML as part of the build.
• Search engines are likely to index stub pages, which potentially dilutes SEO value of the moved pages. To offset this we should ensure that the actual content is not duplicated in the stub page and the new / moved page.
• URLs for stub pages maintain a 200 status instead of a 301 status. We can, however, set the preferred url with canonical meta tags in the RST files.

Example stub page

Here is what an RST stub page should look like:

:orphan:

.. meta::
   :canonical: https://docs.ansible.com/ansible/latest/guide/page.html

**********
Page Title
**********

This page has moved to :ref:`moved_page_reference`.
Please update your bookmarks or links with the correct page url.

Background

So why do we need to create stub files in the first place?

Ansible community documentation that is available from docs.ansible.com has long been hosted on Red Hat managed infrastructure. This infrastructue includes Apache httpd services with the mod_rewrite module that provides dynamic redirect functionality. When pages in the Ansible package documentation were relocated or removed, a redirect rule was added to the .htaccess configuration file. These files are sourced in the ansible/docsite repository.

To migrate hosting to ReadTheDocs, and provide greater access to community maintainers, a strategy is needed to prevent broken redirects that would result in 404s and degradation of SEO authority. We can create some redirects in the ReadTheDocs project but there is a limit of 100 redirects per project.

Unfortunately the number of existing redirects already exceeds that limit. For moved pages alone, we have more than 200 redirects. Read The Docs imposes a limi of 100 redirects per project.

Additionally, a goal is to move away from creating redirects because this adds a lot of maintenance overhead to the project. The reasoning for the creation of many of the redirect rules in the .htaccess configuration files is historical knowledge with little no documentation.

Looking in the .htaccess file you can see rules such as this one:

RedirectMatch permanent "^/ansible/(devel|latest)/user_guide/playbooks_blocks.html" "/ansible/$1/playbook_guide/playbooks_blocks.html"

When we migrate the docs.ansible.com subdomain to Read The Docs hosting, we can update these redirects to point to the new project as follows:

RedirectMatch permanent "^/ansible/(devel|latest)/user_guide/playbooks_blocks.html" "https://docs.ansible.com/projects/ansible/$1/playbook_guide/playbooks_blocks.html"

This redirect works for any links that are internal to the Ansible community documentation. For instance, if a page references user_guide/playbooks_blocks.html the redirect will point to the corresponding page on ReadTheDocs.

However, if there is an external link such as a bookmark or reference from a third-party site, the redirect does not take effect because the docs.ansible.com subdomain has moved to ReadTheDocs.

In other words, the following page will not be on Read The Docs and the redirect does not take effect because docs.ansible.com is served by Read The Docs:

https://docs.ansible.com/ansible/latest/user_guide/playbooks_blocks.html

As a result, any external link that points to that page will cause a 404 error.

Stub pages on Read The Docs

One thing to note is that, as long as docs.ansible.com is on Red Hat managed infrastructure, we can keep the redirects in place. We do want to create all the stub pages before we migrate the subdomain over to RTD but they don't really serve a purpose until after the migration.

On ReadTheDocs we will have a top-level redirect in place that handles the new URL structure with the /projects/ subfolder:

Type: Exact Redirect
From URL: /ansible/*
To URL: /projects/ansible/:splat
Force Redirect: True

Here is how that top-level redirect will work:

# Original url
https://docs.ansible.com/ansible/latest/user_guide/playbooks_blocks.html

# Target url after redirection
https://docs.ansible.com/projects/ansible/latest/user_guide/playbooks_blocks.html

In this case, the stub page is in place for the target url. You can actually view the user_guide/playbooks_blocks.html stub page on ReadTheDocs today.

We created that stub page when we restructured the user guide. We also created the redirects for docs.ansible.com but, again, we won't be moving the redirects over to Read The Docs because there are too many.