Modules

0000-modules/README.md

@@ -0,0 +1,34 @@
+- Feature Name: modules
+- Start Date: 2017-08-07
+- RFC PR: (leave this empty)
+- Rust Issue: (leave this empty)
+
+# Summary
+[summary]: #summary
+
+This is a redesign of the Rust module system, intended to improve its
+ergonomics, learnability, and locality of reasoning. Because this is a
+relatively large proposal, it has been broken into multiple text files.
+
+# Table of Contents
+
+* **[Motivation][motivation]** - why we propose to make this change
+* **[Overview][overview]**  - a high level overview of what it will be like to
+use the new system as a whole
+* **Detailed design** - the details of the proposal, broken into multiple
+sections:
+    * **[Loading Files][loading-files]**
+    * **[The `local` keyword][local]**
+    * **[Use, mod, and export][use-mod-export]**
+* **[Migration][migration]** - this proposal involves migrating from one system
+to another, and this section describes it in detail.
+
+Each of the detailed design subsections contains its own description of
+drawbacks and alternatives.
+
+[motivation]: motivation.md
+[overview]: overview.md
+[loading-files]: detailed-design/loading-files.md
+[local]: detailed-design/local.md
+[use-mod-export]: detailed-design/use-mod-export.md
+[migration]: migration.md

0000-modules/detailed-design/loading-files.md

@@ -0,0 +1,231 @@
+# Loading Files
+
+When building a Rust project, rustc will load and parse some files as Rust code
+in addition to the root module. These will be used to construct a module tree.
+By default, cargo will generate the list of files to load in this way for you,
+though you can generate such a list yourself and specify it in your
+`Cargo.toml`, or you can generate the list in another way for your non-cargo
+build system.
+
+This eliminates the need to write `mod` statements to add new files to your
+project. Instead, files will be picked up automatically as a part of your
+module hierarchy.
+
+## Detailed design
+
+### Processing the `--modules` list (rustc)
+
+rustc takes a new argument called `modules`, which takes a space separated list
+of files. Each file will be treated as a module, and rustc will attempt to open
+and parse every file listed, reporting errors if it is unable to. It will mount
+these files as a tree of Rust modules using rules which mirror the current
+rules for looking up modules.
+
+It will not attempt to open or parse files where:
+
+* The file name is not a valid Rust identifier followed by `.rs`.
+* The file is not in a subdirectory of the directory containing the root
+module.
+* Any of the subdirectories of the root module in the path to this file are not
+valid Rust identifiers.
+
+Cargo's default system will not pass any files that would be ignored by these
+conditions, but if they are passed by some other system, they are ignored
+regardless. For example, in a cargo managed crate with no dependencies, this
+would be a valid way to invoke rustc by hand:
+
+```
+rustc src/lib.rs --modules src/*.rs src/**/*.rs
+```
+
+Rust will mount files as modules using these rules:
+
+* If a file is named `mod.rs`, it will mount it as the module for the name of
+directory which contains it (the directory containing the root module cannot
+contain a `mod.rs` file; this is an error).
+* Otherwise, it will mount it at a module with the name of the file prior to
+the `.rs`.
+
+All modules mounted this way are visible to the entire crate, but are not (by
+default) visible in the external API of this crate.
+
+If, during parsing, a `mod` statement is encountered which would cause Rust to
+load a file which was a part of the `--modules` list, this statement will be
+used to control the visibility of that module. If the module was not a part of
+the `--modules` list, it will be loaded in the same way that it is loaded
+today.
+
+If a module is mounted multiple times, or there are multiple possible files
+which could define a module, that continues to be an error.
+
+Another result of this design is that the naming convention becomes slightly
+more flexible. Prior to this RFC, if a module file is going to have submodule
+files, it must be located at `mod.rs` in the directory containing those
+submodules - e.g. `src/foo/mod.rs`. As a result of this RFC, users can instead
+locate it at `src/foo.rs`, but still have submodules in the `foo` directory.
+Some users have requested this functionality because their tooling does not
+easily support distinguishing files with the same name, such as all of their
+`mod.rs` files.
+
+In fact, in this design, it is not necessary to have a `foo.rs` or `foo/mod.rs`
+in order to have modules in the `foo` directory. Without such a file, `foo`
+will just have no items in it other than the automatically loaded submodules.
+For example:
+
+```
+/foo
+    bar.rs
+    baz.rs
+lib.rs
+```
+
+This mounts a submodule `foo` with two items in it: submodules `bar` and `baz`.
+There is no compiler error.
+
+#### The `#[ignore]` attribute
+
+Additinally, modules can be annotated with the `ignore` attribute. This
+attribute will be treated as a kind of unsatisfiable cfg attribute - a module
+tagged `#[ignore]` will not be compiled.
+
+The ignore attribute can take any number of attribute arguments, which are
+paths. These are relative paths to items (usually modules) which should be
+ignored. Without an argument, `#[ignore]` is `#[ignore(self)]`. But you could
+also write:
+
+```rust
+#![ignore(foo, bar::baz)]
+```
+
+To ignore both `foo` and `bar::baz` submodules of this module, and all of their
+submodules.
+
+### Gathering the `--modules` list (cargo)
+
+#### Library and binary crates
+
+When building a crate, cargo will collect a list of paths to pass to rustc's 
+`--modules` argument. It will only gather files for which the file name
+has the form of a valid Rust identifier, followed by the characters `.rs`.
+
+cargo will recursively walk the directory tree, gathering all appropriate
+files, beginning with the directory which contains the crate root file. It will
+ignore these files and directories:
+
+* The crate root file itself.
+* Any directory with a name which is not a valid Rust identifier.
+* If the crate root is in the `src` subdirectory of the Cargo manifest
+directory, and there is a directory called `src/bin`, cargo will ignore that
+subdirectory.
+
+In short, cargo will include all appropriately named files inside the directory
+which contains the crate root, except that it will ignore the `src/bin`
+directory.
+
+Packages containing multiple crates which wish to use the default module list
+will need to make sure that they do not have multiple crates rooted in the same
+directory, or within a subdirectory of another crate. The most likely
+problematic crates today are those which have both a `src/lib.rs` and a
+`src/main.rs`. We recommend those crates move their binary crate to the
+`src/bin` directory solution.
+
+While gathering the default module list, cargo will determine if any other
+crate is rooted in a directory which would be collected by the default module
+list, and will instead not pass a `--modules` list and issue a warning in
+that case, informing users that they need to rearrange their crates or provide
+a list of modules themselves.
+
+(**Note:** These projects will receive a warning, but will not be broken,
+because the `mod` statements they already contain will continue to pick up
+files.)
+
+#### Tests, examples, and benchmarks
+
+Test, example, and benchmark crates follow a different set of rules. If the
+crate is located in the appropriate top-level directory (`tests`, `examples`,
+and so on), no `--modules` list will be collected by default. However,
+subdirectories of these directories will be treated as individual binary
+crates: a `main.rs` file will be treated as the root module, and all other
+appropriately named files will be passed as `--modules`, using the same
+rules described above.
+
+So if you have an examples directory like this:
+
+```
+examples/
+    foo.rs
+    bar/
+       main.rs
+       baz.rs
+```
+
+This contains two examples, a `foo` example and a `bar` example, and the `bar`
+crate will have `baz.rs` as a submodule.
+
+The reason for this is that today, cargo will treat every file in `tests`,
+`examples`, and `benches` as independent crates, which is a well-motivated
+design. Usually, these are small enough that a single file makes sense.
+However, today, cargo does not make it particularly easy to have tests,
+examples, or benchmarks that are multiple files. This design will create a
+pattern to enable users to do this.
+
+#### The `load-modules` target flag
+
+Target items in the Cargo.toml have a `load-modules` flag, which is set to true
+by default. Setting it to false causes cargo not to pass a `--modules` list at
+all.
+
+For example, a crate with just a library that does not want cargo to calculate
+a modules list would have a toml like this:
+
+```toml
+[package]
+name = "foo"
+authors = ["Without Boats <[email protected]>"]
+version = "1.0.0"
+
+[lib]
+load-modules = false
+```
+
+In practice, setting this flag to false will make mod statements necessary for
+loading additional files in the project.
+
+## Drawbacks
+
+The RFC authors believe that making mod statements unnecessary is a *net* win,
+but we must acknowledge that it is not a *pure* win. There are several
+advantages that mod statements bring which will not be fully replicated in the
+new system.
+
+Some workflows have been convenienced by the fact that statements need to be
+added to the source code to add new modules to files. For example, it makes it
+easier for users to leave their src directories a little bit dirty while
+working, such as through an incomplete `git stash`. If users wish to comment
+out a module, it can be easier to comment out the `mod` statement than to
+comment out the module file. In general, it enables users to leave code which
+would not compile in their src directory without explicitly commenting it out.
+
+Some users have expressed strong concerns that by deriving the module structure
+from the file system, without making additional syntactic statements, they will
+not be able to as easily find the information they need to navigate and
+comprehend the codebases they are reading or working on. To partly ease their
+concern, the RFC allows users to explicitly specify their module lists at the
+build layer, instead of the source layer. This has some disadvantages, in that
+users may prefer to not have to open the build configuration either.
+
+This will involve migrating users away from `mod` statements toward the new
+system.
+
+## Alternatives
+
+An alternative is to do nothing, and continue to use `mod` statements.
+
+We could also put the file-lookup in rustc, instead of cargo, and have rustc
+perform its own directory walk. We believe this would be a bad choice of
+layering.
+
+During the design process, we considered other, more radical redesigns, such as
+making all files "inlined" into their directory and/or using the filename to
+determine the visibility of a module. We've decided not to steps that are this
+radical right now.

0000-modules/detailed-design/local.md

@@ -0,0 +1,174 @@
+# The `local` keyword
+
+This RFC introduces a `local` keyword, which means "local to this crate." This
+keyword is used in two contexts:
+
+* It is a new visibility, equivalent to today's `pub(crate)`.
+* The items inside of this crate are located at an absolute path under the
+special `local` module, instead of at the absolute root.
+
+We make a handful of other changes to paths and visibility to incorporate this
+new keyword into the system smoothly.
+
+## Detailed design
+
+### Changes to absolute paths
+
+As a result of [RFC #2088][extern-crate], `extern crate` will be deprecated. In
+conjunction with that RFC, all `--extern` dependencies are mounted at the
+absolute path root. However, that no longer corresponds to the root file of
+this crate (usually `lib.rs` or `main.rs`). Instead, that root *contains* the
+root file of this crate in a module named `local`.
+
+`use` statements continue to take paths from the absolute root, but this no
+longer corresponds to the crate root. Other paths continue to be relative by
+default, but can be made absolute with an initial `::`.
+
+The use of the `local` keyword has several advantages:
+
+1. Users can always tell whether a path comes from an external dependency or
+from within this crate. This increases the explicit, local information. Today
+you have to know if `::foo` refers to a top-level module or a dependency.
+2. It is no longer the case that within the root module, relative paths and
+absolute paths correspond. This correspondence has led users to believe that
+they will always correspond, causing significant confusion when they don't
+correspond in submodules.
+3. Our survey of the ecosystem suggests that external imports are more common
+than internal imports and we believe this will continue to be true for the
+reasons listed below. Thus if we are going to distinguish external dependencies
+from internal ones, it makes more sense to put the distinguishing mark on
+internal dependencies.
+    * We encourage users to modularize their code using the `crates.io`
+    ecosystem, leading to a proliferation of external dependencies.
+    * Many large projects are broken into several crates, making even
+    intra-project dependencies "external" dependencies for the purposes of
+    paths.
+4. Several other languages work this way; most use the name of this package as
+the distinguisher, rather than a keyword, but a keyword has these advantages:
+    * If a crate is renamed, you do not need to rename all internal imports.
+    * Crates currently can depend on other crates with the same name; this is
+    commonly employed when you want a library with the same name as a binary
+    (for example, cargo). Using a keyword avoids breaking this pattern.
+
+#### Special path prefixes
+
+The special path prefixes are `local`, `super`, and `self` (`super` and `self`
+are implemented today). These prefixes are supported in both relative and
+absolute paths. They can be prefixed with the `::` absolute root in either
+context. That is, all of these are valid forms:
+
+```rust
+local::module::Item;
+super::sibling::Item;
+self::child::Item;
+
+::local::module::Item;
+::super::sibling::Item;
+::self::child::Item;
+
+use local::module::Item;
+use super::sibling::Item;
+use self::child::Item;
+
+use ::local::module::Item;
+use ::super::sibling::Item;
+use ::self::child::Item;
+```
+
+### Changes to visibility
+
+The `local` visibility is very similar to `pub`, except that it is not exported
+in the public API of the crate. This gives it the same semantics as
+`pub(crate)` has today.
+
+Local, however, can take restrictions, using the same parenthical syntax
+suppoted on `pub` today:
+
+* `local(self)` - visible to this module and its children
+* `local(super)` - visible to this module's parent and its children
+* `local(crate)` - visible to this crate; the `crate` modifier is redundant,
+and is linted against by dead code (but supported, for possible use in macros).
+* `local(in $path)` - visible to everything under the given path, which must be
+a parent of this module (such as `local(in super::super)`
+
+Visibility restrictions on `pub` are deprecated in favor of using visibility
+restrictions on `local` instead. The result of this is that `local` always
+means locally visible (with modifiers giving different definitions of `local`),
+while `pub` always means visible in the public API of this crate.
+
+#### The private `pub` lint
+
+We also introduce a lint, called `nonpublic_pub` which is error by default when
+compiling a library, and allow by default when compiling a binary. The
+`nonpublic_pub` lint checks that all items marked `pub` are actually, somehow,
+accessible in the public API. This could mean that they are accessible at their
+canonical location (where they are defined), but it could also mean that they
+are re-exported publically at another point if the module they are in is not
+public.
+
+For example, this would trigger the lint, because there is no public path to
+`Foo`:
+
+```rust
+mod foo {
+    pub struct Foo;
+}
+```
+
+But either of these would be fine:
+
+```rust
+pub mod foo {
+    pub struct Foo;
+}
+
+// Foo is public at ::foo::Foo
+```
+
+```rust
+mod foo {
+    pub struct Foo;
+}
+
+pub export foo::Foo;
+
+// Foo is public at ::Foo
+```
+
+
+## Drawbacks
+
+This section of the RFC deprecates the current absolute path syntax for local
+dependencies, as well as the current pub(restricted) syntax. This results in
+some churn as users have to migrate to the new syntax. However, it does follow
+the standard epoch deprecation procedures.
+
+The `local` keyword is not a perfect fit, because it is plausible to believe it
+means something more narrow than "local to this crate" - like local to this
+module, or local to this function. However, it has some narrow constraints that
+make it hard to find a better alternative.
+
+## Alternatives
+
+We've considered multiple alternative keywords for this purpose, instead of
+`local`, such as `protected`, `internal`, `crate`, `lib` or a shorthand like
+`loc` or `vis`. There are some constraints though:
+
+* Shorter tends to be better than longer.
+* It needs to be sensible as the prefix to a path (making sense as a spatial
+designator is preferable.)
+* It needs to be sensible as a visibility (an adjective is strongly
+preferable).
+* It can't be an existing crate on crates.io (unless its been reserved for this
+purpose, as `local` has been).
+* It needs to have support for the `(restricted)` syntax.
+
+Based on these criteria, we've chosen `local`, though we aren't wedded to it
+and are open to an alternative which fits these criteria better.
+
+There is an overall alternative, proposed in a previous RFC, in which `pub`
+means what `local` does in this RFC, and `export` means "part of the public
+API." This path was not pursued, because it is a more radical and complex
+breakage.
+
+[extern-crate]: https://github.com/rust-lang/rfcs/pull/2088

0000-modules/detailed-design/use-mod-export.md

@@ -0,0 +1,126 @@
+# Use, mod, and export
+
+## Detailed design
+
+### The `use` statement
+
+Like implementation blocks, `use` statements are not "items" in the sense that
+they do not create new names in the item hierarchy, and for that reason do not
+have their own visibility modifier.
+
+Instead, `use` statements just bring names into scope for the resolution of
+paths within the same scope that they are declared in (whether that be module,
+function, etc), without creating declared items of those names at this point in
+the module hierarchy.
+
+`use` statmeents with a visibility (such as `pub use`) are deprecated, and
+imports that have taken advantage of the item-like behavior of use (such as
+`use super::*` picking up imports in the parent module) will receive warnings
+that can be fixed by explicitly importing the items picked up this way.
+
+### The `mod` statement
+
+In this system, the `mod` statement still exists; it is primarily used as a
+way to control the visibility of submodules.
+
+If a module is not in the `--modules` list, it will still be added by `mod`
+statements, just as it would be today. But if it is in the modules list, the
+mod statement can be used instead to control the visibility of the module. For
+example:
+
+```rust
+pub mod foo;
+local(super) mod bar;
+local(self) mod baz;
+```
+
+If a module is in the `--modules` list, and the user has declared it with a
+`local` visibility, this is redundant (as `--modules` loaded modules are
+`local` inherently). A lint warns on dead code for `local` modules that are
+already loaded from the `--modules` list, the `redundant_mod_statements`.
+
+#### Cfg attributes on `mod` statements
+
+This RFC specifies that cfg attributes on mod statements will be treated
+analogously to `#!` attributes on the inside of modules. There is a
+possibility, in conjunction with the changes to how files are loaded, that
+these attributes will have surprising behavior.
+
+For example, consider this code:
+
+```rust
+#[cfg(test)]
+local mod tests;
+```
+
+In a naive implementation, this could remove the mod statement, but because the
+module is loaded through the `--modules` list, the module itself is still
+loaded, even outside of the test cfg. This RFC specifies that this cfg
+attribute is treated as applied to *the module*, not just this statement.
+
+### The `export` statement
+
+The `export` statement is used for re-exporting items located at a different
+point. It takes a path, which, like all paths other than those in `use`
+statements, is a relative path by default. `export` replaces `pub use` for this
+purpose:
+
+```rust
+pub export foo::Bar;
+local export foo::Baz;
+```
+
+It is an error to export an item at a greater visibility than its declared
+visibility on its canonical declaration.
+
+The reason to introduce this, instead of continuing to use `pub use`, is that
+it takes a relative path, which is usually what users want for re-exports, and
+it allows `use` to be simpler (not have item semantics, not possible to take a
+visibility), leading to a more incremental learning process.
+
+### Required visibility on `mod` and `export`
+
+Because the `mod`-semicolon statement (not `mod` with a block) and the `export`
+statement are both about configuring the visibility of an API, they will both
+require a visibility statement, rather than having a default visibility. For
+example, these will become errors:
+
+```rust
+mod foo;
+export foo::Bar;
+```
+
+And you will have to specify their visibilities, as in:
+
+```rust
+local(self) mod foo;
+pub export foo::Foo;
+```
+
+Possibly after use under the new system shows certain visibilities are
+overwhelmingly common, we can add a default visibility during the next
+checkpoint or later.
+
+On the current checkpoint, bare `mod` statements will receive a warning. They
+will have the exact same behavior, with no warning, by amending them to
+`local(self)`.
+
+## Drawbacks
+
+The main drawbacks of this specific section of the RFC are that it deprecates
+several things:
+
+* Use statements as items and `pub use` statements are deprecated
+* Mod statements without visibilities are deprecated
+
+It also introduces a new form, `export`, which today is just a "use statement
+with a visibility." While this change has motivation, it comes at the cost of
+some churn.
+
+## Alternatives
+
+The changes here are specifically designed to interact well with the other
+changes to how files are loaded and the new `local` visibility. The main
+alternative that only impacts this part of the proposal would be to not
+introduce the `export` statement, and instead keep the current behavior of `pub
+use`.

0000-modules/migration.md

@@ -0,0 +1,51 @@
+# Migration
+
+The changes proposed by this RFC are significant breaking changes to the
+language. To complete them entirely we will need to pass a checkpoint/epoch
+boundary as described in [RFC #2052][checkpoints]. However, we will begin with
+a multi-phase deprecation process to ease the transition.
+
+We will also introduce a tool that will not only automatically move a crate
+onto a new system, but also *improve clarity* of the user's code by
+intentionally drawing the distinction between the `pub` and `local`
+visibilities that today does not firmly exist.
+
+## The automatic upgrade path
+
+Concurrent with the first release under this RFC, we will also release a tool
+that eliminates all deprecated forms from the user's codebase and moves them
+into the preview checkpoint. It will also automatically fix any other errors
+introduced by the checkpoint transition.
+
+This tool will be distributed in such a way that an upgrade to the first
+version that provides it puts it on the user's system. If users are willing to
+use this tool, all they will need to do is run it and they will get code
+without any deprecation errors. This is the easiest and the recommended way
+to upgrade to the new checkpoint and apply these module changes.
+
+This tool will also correctly distinguish between items which need to be marked
+`pub` and items which should be marked `local`, and mark those items
+appropriately.
+
+## The manual upgrade path
+
+For users who wish to upgrade manually, the following new features will be
+introduced on the current (2015) checkpoint:
+
+* The `local` and `export` features will be provided as contextual keywords.
+* The `--modules` API for rustc will be introduced.
+* cargo will accept the `load-modules` flag, but it will be set to false by
+default.
+
+The following deprecations will be introduced (possibly staged over multiple
+releases):
+
+* `use` with a visibility
+* Imports taking advantage of `use`'s item nature
+* Restriction modifiers on `pub` (as opposed to `local`)
+* `mod` statements without a visibility
+* Absolute local paths without the `local::` prefix
+
+In the new epoch, all of these will become hard errors and the `load-modules`
+flag in cargo will be set to `true` by default, turning on the module loading
+system.

0000-modules/motivation.md

@@ -0,0 +1,146 @@
+# Motivation
+
+## Background
+
+For the past several months, we have been investigating the module system, its
+weaknesses, strengths, and areas of potential improvement. Two blog posts have
+been produced, which lay out a part of the argument in favor of changing the
+module system:
+
+* [The Rust module system is too confusing][too-confusing] by withoutboats
+* [Revisiting Rust's modules][revisiting] by aturon
+
+Both of these posts contain proposals for how to revamp the module system,
+neither of which are representations of what this RFC contains. However, they
+provide valuable background on the conversation that has been occurring, since
+January of this year, about how the Rust module system could be improved.
+
+Fundamentally, we believe that the current module system is difficult to learn
+how to use correctly, and contains unnecessary complexity which leaves new
+users confused and advanced users annoyed. We have collected empirical data in
+support of that belief, and also have formed a model of the underlying problems
+with the system that we hope to resolve or mitigate with this proposal.
+
+Its important to keep this in mind: our contention is not that the module
+system is the most difficult part of the language, only that it is
+**unnecessarily** difficult. Some aspects of Rust have high inherent novelty -
+such as ownership & lifetimes - and cannot be just *made* easier (though the
+syntax could always be more obvious, etc). We do not believe this is true of
+our modules, which provide similar benefits to systems in many mainstream
+languages which do not have such a problem.
+
+## Evidence of a problem
+
+### Survey data
+
+In the survey data collected in 2017, ergonomics issues were one of the major
+challenges for people using Rust today. While there were other features that
+were raised more frequently than the module system (lifetimes for example), we
+don't feel that modules ought to be considered an ergonomics problem at all.
+
+Here are some quotes (these are not the only responses that mention the module
+system):
+
+> Also the module system is confusing (not that I say is wrong, just confusing
+> until you are experienced in it).
+
+> a colleague of mine that started rust got really confused over the module
+> system
+
+> You had to import everything in the main module, but you also had to in
+> submodules, but if it was only imported in a submodule it wouldn't work. 
+
+> I especially find the modules and crates design weird and verbose
+
+> fix the module system
+
+One user even states that the reason they stopped using Rust was that the
+"module system is really unintuitive." Our opinion is that if any user has
+decided to *stop using Rust* because they couldn't make modules work, the
+module system deserves serious reconsideration.
+
+### Found feedback
+
+@aturon devoted some time to searching for feedback from Rust users on various
+online forums about Rust (GitHub, the discourse forums, reddit, etc). He found
+numerous examples of users expressing confusion or frustration about how the
+module system works today. This data is collected in [this
+gist][learning-modules].
+
+## Underlying problems
+
+### Lots of syntax, all of it frontloaded
+
+The current Rust module system has four primary keywords: `extern crate`,
+`mod`, `use`, and `pub`. All of these are likely to be encountered very early
+by users - even if you're working on a small binary project, just to learn, you
+are likely going to want to add a dependency and break your code into two
+files. Doing those two things introduces the need to use  all four keywords
+that are a part of the module system, and to understand what each of them does.
+
+In our proposal, there are three primary keywords: `use`, `pub`, and `export`.
+And in an early project, only `use` and `pub` are going to be necessary. Not
+only that, the `export` keyword is just a visibility, like `pub`, and can be
+viewed as an extension of the pre-existing knowledge about visibility learned
+when your crate gains an external API.
+
+We believe we will simplify the system by requiring users to learn and do less
+stuff. 
+
+We also believe reducing the syntax improves the ergonomics for advanced users
+as well. Many of these declarations tend to feel like 'boilerplate' - you write
+the only thing you could possibly write. You may forget to write a `mod`
+statement, leading to a compiler error (worsening your edit-compile-debug
+cycle). We, the RFC authors, frequently have this experience.
+
+## Path confusion
+
+A problem, recognized by many users, is a confusion that exists about how paths
+work in Rust. Though this has often been glossed as confusion about the
+difference between absolute and relative paths, we don't believe this is true.
+Very many languages use absolute paths for their import statement, but relative
+paths internally to a module, without demonstrating the same confusion. We
+believe the confusion has two causes, both addressed by this RFC.
+
+### Differences between crate root and other modules
+
+In the crate root, absolute and relative paths are the same today. In other
+languages this tends not to be the case. Instead, a common system is to place
+the root of the current package under some namespace; this RFC proposes to do
+just that, using the new `crate` namespace.
+
+This will make absolute paths different from relative paths in every module,
+including the crate root, so that users do not get mislead into believing that
+`use` statements are relative, or that all paths are absolute.
+
+### Multiple keywords import things
+
+Another problem with the current system is that `extern crate` and `mod` bring
+names into scope, but this is not immediately obvious. Users who have not
+grasped this can be confused about how names come into scope, believing that
+dependencies are inherently in scope in the root, for example (though it is
+actually `extern crate` which brings them into scope). Some users have even
+expressed confusion about how it seems to them that `extern crate` and `mod`
+are "just" fancy import statements that they have to use in seemingly arbitrary
+cases, not understanding that they are using them to construct the hierarchy
+used by the `use` statements.
+
+We solve this problem by removing both the `extern crate` and `mod` statements
+from the language.
+
+## Nonlocal reasoning
+
+Another frustration of the current module system - which affects advanced users
+at least as much as newer users - is the way that visibility is highly
+nonlocal. When an item is marked `pub` today, its very unclear if it is
+actually `pub`. We make this much clearer by the combination of two choices in
+this RFC:
+
+- All items that are `export` (equivalent of today's `pub`) must *actually* be
+  exposed in the external API, or the user receives an error.
+- All modules are crate visible by default, so every visibility less than
+  `export` is inherently self-descriptive.
+
+[too-confusing]: https://withoutboats.github.io/blog/rust/2017/01/04/the-rust-module-system-is-too-confusing.html
+[revisiting]: https://aturon.github.io/blog/2017/07/26/revisiting-rusts-modules/
+[learning-modules]: https://gist.github.com/aturon/2f10f19f084f39330cfe2ee028b2ea0c

0000-modules/overview.md

@@ -0,0 +1,219 @@
+# Overview of the new system
+
+This is intended to be a "guide-level" overview of the new module system, to
+give readers a sense of how things will work after this RFC is fully
+implemented.
+
+This document is divided into two sections: one about structuring your project,
+and the other about crafting your API. This distinction is intentional: more
+users contribute to binary projects than to libraries, and it is especially
+true that new users are more likely to start with binaries than libraries.
+Binaries do not have significant public APIs. Libraries, in contrast, care that
+their public APIs are well-structured & also stable across versions.
+
+This system is intended to be incremental - users contributing to binary
+projects only need to worry about half of the system, whereas the second half
+can be reserved to users contributing to libraries. The first section, about
+internal project structure, applies to both binaries and libraries. The second
+section applies only to users writing libraries.
+
+## Structuring your project
+
+### Adding an external dependency to your project
+
+To add a new dependency to a project managed with cargo, users need to edit
+their `Cargo.toml`. This is the only thing they need to do to make that
+dependency available to them:
+
+```toml
+[dependencies]
+serde = "1.0.0"
+```
+
+Once a user has done this, they will be able to use anything from that
+dependency's public API in their project, using the `use` statement or absolute
+paths. These paths look exactly like they look today:
+
+```rust
+use std::collections::HashMap;
+use serde::Serialize;
+::std::iter::Iterator
+::serde::de::DeserializeOwned
+```
+
+The major difference from today is that `extern crate` declarations are no
+longer necessary. This change however is covered in [RFC #2088][extern-crate],
+not directly by this RFC.
+
+### Adding a new file to your crate
+
+When a user wants to break their project into multiple files, they can do so by
+creating a new `.rs` file in the directory that contains their crate root (or a
+subdirectory thereof). For most users, this would be the `src` directory.
+
+This file will automatically be picked up to create a new **module** of the
+user's crate. The precise mechanism is discussed in [another
+section][loading-files] of the RFC. In brief, cargo will find the file and tell
+rustc to treat it as a new module using an interface that is also available to
+non-cargo users. Cargo users who do not want this behavior to be automatic can
+instead specify their modules using a field in their `Cargo.toml`.
+
+The major difference from today is that a `mod` statement is no longer
+necessary to get rustc to look up and load your files. The `mod` keyword still
+exists to support inline modules (`mod foo { ... }`).
+
+### Making items visible to the rest of your crate
+
+By default, all modules are visible to the rest of the crate. However, all
+items inside them - like types, functions, traits, etc - are private to that
+module. To make them visible to outside of the module, users add a visibility
+modifier. There are two visibility modifiers in the future Rust:
+
+* `pub` - this means them item is visible in the public API of this crate
+* `local` - this means the item is visible only inside this crate. It can take
+a modifier to restrict it even further, like `local(super)`.
+
+For users writing binaries, the difference between `pub` and `local` does not
+matter. For users writing libraries, `pub` items which are not actually
+accessible from outside the crate will be linted against.
+
+```rust
+// Private to this module
+struct Foo;
+
+// Publicly visible
+pub struct Bar;
+```
+
+There are two significant differences from today:
+
+* All modules which are automatically loaded are public to the rest of the
+crate, rather than taking a visibility modifier.
+* The new `local` visibility is added, equivalent to `pub(crate)`, and
+restricted visibilites are moved to the `local` visibility instead of `pub`.
+
+### Importing items from other parts of your crate
+
+Once a user has a public item in another module, they can import it using the
+`use` statement, just like items from external dependencies. Paths to items in
+modules of this crate work like this:
+
+* All these paths begin with the `local` keyword, because they are local paths.
+* All modules in the `src` directory are mounted in the root module, all
+modules in other directories are mounted inside the module for that directory.
+* Items are mounted in the module that defines them.
+
+So if a user has a structure like this:
+
+```
+// src/foo.rs
+pub struct Bar;
+```
+
+They can access it with `use` and absolute paths like this:
+
+```rust
+use local::foo::Bar;
+::local::foo::Bar;
+```
+
+The major difference from today is that the `local` keyword is used to prefix
+items inside this crate, so that users can distinguish external dependencies
+from modules in this crate by only looking at the import statement.
+
+## Crafting your API
+
+### Exporting things from your crate
+
+When a user writing a library wants to declare something exported from their
+crate, they can use the `pub` visibility modifier to declare it as a part of
+their public API:
+
+```rust
+// This is exported
+pub struct Foo;
+```
+
+Items marked `pub` inside of files which are not public are not visible outside
+of this crate, because the full path to that item is not `pub`. Instead, they
+need to provide a public mod statement to make that module public:
+
+```rust
+// In `src/lib.rs` Imagine that there is a `src/foo.rs` as well.
+pub mod foo;
+```
+
+Mod statements exist in this system to control the visibility of modules. If
+the `local` visibility is sufficient for your use cases, you do not need to use
+`mod` statements.
+
+### Hiding items that shouldn't be exported
+
+In a library, items which are marked `pub` must be publicly accessible in the
+library's API; a lint warns users when they are not. For items which are not
+intended to be a part of the API, the `local` visibility makes them visible
+only inside the library:
+
+```rust
+// only visible "locally" - inside this crate
+local struct Foo;
+```
+
+Even in a public module, a local item won't be visible in the public API. The
+local visibility can also take restrictions, to make it more restricted than
+the default localness, such as:
+
+```rust
+// Only visible in the parent module
+local(super) struct Foo;
+```
+
+The `local` visibility takes the same role that `pub(restricted)` takes in the
+current system.
+
+### Re-exporting items to simplify your public API
+
+Sometimes users want to make items public using a path hierarchy that's
+different from the true path hierarchy which makes up the crate. This can be to
+simplify the API, or to maintain backwards compatibility. Users can do this by
+using the `export` keyword with a relative path (just like all non-`use`
+paths):
+
+```rust
+local mod foo;
+local mod bar;
+
+pub export foo::Foo;
+pub export bar::Bar;
+```
+
+This will create a new public path to the item, even if the canonical path to
+the item is not public. The item itself needs to be marked `pub`, or this is an
+error.
+
+This replaces the functionality of `pub use`.
+
+## Opt-outs
+
+There are two primary mechanisms for opting out of this system: one, for
+temporarily avoiding compiling a file that would otherwise be compiled, during
+development, and the other, for making mod statements with visibility mandatory
+for your project.
+
+### The `#[ignore]` attribute
+
+If you want a file to be ignored, you can use the `#[ignore]` attribute on the
+top of it. This file will not be compiled and you will not get errors from it
+unless they were very early parsing errors. That is, this file will not need to
+pass name resolution, typechecking, or ownership & borrow checking. We will
+make a best effort to make the ignore attribute read as early as possible.
+
+### The `load-files` directive
+
+If you want module statements to always be required to load files in a crate,
+every target section of the Cargo.toml has a `load-files` directive, which can
+be set to false. This will prevent cargo from telling Rust about module files,
+making `mod` statements necessary to find files.
+
+[extern-crate]: https://github.com/rust-lang/rfcs/pull/2088
+[loading-files]: detailed-design/loading-files.md