Skip to content

Commit

Permalink
Introduce the modern Stdlib API; Fix re-exports (#1727)
Browse files Browse the repository at this point in the history
This PR fixes all the rough edges around re-exports and re-exports with
aliases, and specifically re-exporting namespaces.

## Summary of Changes

* The package store has to be available to the HIR and FIR, because a
re-export can refer to an item from another package. To resolve any
`ItemId` where `package` is not `None`, we need to query the
`PackageStore`. So that got plumbed all the way through.
* An export can be its own `ItemKind`. It serves as a "reference" to
another item in another package if it is present. This is used to ensure
that items defined in package A will show up in package B's items, if it
is re-exported from package B.
* In both the resolver and the type checker, we follow the "chain of
re-exports" until we find the original item that is being exported.
There are multiple places that this happens. [Here is an
example](https://github.com/microsoft/qsharp/pull/1727/files#diff-d98d007bbe0a3fce75f8d49278e0b566c9f8d91731f3e451296db14bebff001fR403).
* Lots of tests.
* The most nuanced thing is the re-creation of the namespace tree when
adding an external package in the resolver. This was actually the bug
that took me the longest to sort out. I'll break out of this list for
the sake of formatting, as this one requires more explanation:

### The Nuanced Bit

Namespace trees are not acyclic, and the same underlying node can show
up in different places of the tree. For example, in the new standard
library, `Microsoft.Quantum.Arrays` is accessible via both
`Microsoft.Quantum.Arrays` _and_ `Std.Arrays`. When we add an external
package to the local package, we need to recreate this structure.

To accomplish this, I wrote an iterator over the namespace tree which
returns all names for which each namespace can be referred to. For
example, the above namespace would show up in the iterator as
`[Microsoft.Quantum.Arrays, Std.Arrays]`. This allows the resolver to
reconstruct the exact same tree structure and preserve the
aliased/equality property of the two namespaces. And, this is all done
without having to know the underlying `NamespaceId`s. This is important,
because `NamespaceId`s start from 0 for each package compilation -- it'd
be an issue if we had to rely on the IDs of one package to resolve
namespaces in another package.


## Other Notes

With re-exports working as designed, the entire stdlib diff for
re-exporting all items under the new name `Std` is:

```qsharp
namespace Std {
    operation TestFunc() : Unit { Message ("test func"); }
    export TestFunc, Microsoft.Quantum.Arrays, Microsoft.Quantum.Canon, Microsoft.Quantum.Convert, Microsoft.Quantum.Core, Microsoft.Quantum.Diagnostics, Microsoft.Quantum.Logical, Microsoft.Quantum.Intrinsic, Microsoft.Quantum.Math, Microsoft.Quantum.Measurement, Microsoft.Quantum.Random, Microsoft.Quantum.ResourceEstimation, Microsoft.Quantum.Unstable;
}

```

When we support glob exports, in addition to glob imports, this will be
even shorter:

```
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT License.


// This file re-exports the standard library under the name `Std`, which will be the preferred standard library API going forward.

namespace Std {
    export Microsoft.Quantum.*;
}

```
----

## Example


Feel free to test the Bernstein Vaszirani example with the new API:


```qsharp


import Std.Arrays.IndexRange;
import Std.Convert.ResultArrayAsInt;
import Std.Diagnostics.Fact;
import Std.Math.BitSizeI;

@entrypoint()
operation Main() : Int[] {
    let nQubits = 10;

    let integers = [127, 238, 512];
    mutable decodedIntegers = [];
    for integer in integers {
        let parityOperation = EncodeIntegerAsParityOperation(integer);

        let decodedBitString = BernsteinVazirani(parityOperation, nQubits);
        let decodedInteger = ResultArrayAsInt(decodedBitString);
        Fact(
            decodedInteger == integer,
            $"Decoded integer {decodedInteger}, but expected {integer}."
        );

        Message($"Successfully decoded bit string as int: {decodedInteger}");
        set decodedIntegers += [decodedInteger];
    }

    return decodedIntegers;
}

operation BernsteinVazirani(Uf : ((Qubit[], Qubit) => Unit), n : Int) : Result[] {
    use queryRegister = Qubit[n];
    use target = Qubit();

    X(target);

    within {
        ApplyToEachA(H, queryRegister);
    } apply {
        H(target);
        Uf(queryRegister, target);
    }

    let resultArray = MResetEachZ(queryRegister);

    Reset(target);

    return resultArray;
}

operation ApplyParityOperation(
    bitStringAsInt : Int,
    xRegister : Qubit[],
    yQubit : Qubit
) : Unit {
    let requiredBits = BitSizeI(bitStringAsInt);
    let availableQubits = Length(xRegister);
    Fact(
        availableQubits >= requiredBits,
        $"Integer value {bitStringAsInt} requires {requiredBits} bits to be represented but the quantum register only has {availableQubits} qubits"
    );

    for index in IndexRange(xRegister) {
        if ((bitStringAsInt &&& 2^index) != 0) {
            CNOT(xRegister[index], yQubit);
        }
    }
}

function EncodeIntegerAsParityOperation(bitStringAsInt : Int) : (Qubit[], Qubit) => Unit {
    return ApplyParityOperation(bitStringAsInt, _, _);
}

```



----

## Description from #1736 

Yes, that title is a mouthful. This is _quite_ the edge case identified
by @ScottCarda-MS -- thank you for bug bashing!

Re-exports are nuanced. They're basically pointers to `ItemId`s which
point to another package. As a refresher, if an `ItemId` has a package
id (`item_id.package.is_some()`), then it refers to an item outside of
the current package, so it is thusly a reexport. If
`item_id.package.is_none()`, then the item is local to the current
package.

Based on this preexisting notion, re-exports were supported by inserting
`ItemKind::ExportedItem` for re-exports which point to another package.
These are identified in `add_external_package` and inserted into the
names of the user code for resolution. And if an item was local, then no
`ExportedItem` is needed -- we just mark the item's `Visibility` as
public.

But what, what if that local item is exported with an alias? Then, we
need some way to mark it as public _and_ tell the HIR that it has a
different name. To handle this, we insert an `ExportedItem` if an item
is local and the export has an alias.

So, to recap, an `ItemKind::ExportedItem` is inserted if the item being
exported is either from another package (a re-export) or from the local
package, but aliased. Everything mentioned up until now is handled in
#1727.

This PR fixes a bug where a re-export of a local item which is
introduced to the scope via an import alias loses its alias upon
exporting. Wow, what a mouthful. Basically, in order to insert an
`ItemKind::ExportedItem` for an export with an alias, we just checked if
`export.alias.is_some()`. But, actually, you can also basically perform
an aliased export like this: `import Foo as Bar; export Bar;`. In this
case, there is indeed no export on the alias. But we do still need an
`ItemKind::ExportedItem`, because we need to track that this same
`ItemId` is accessible via both names -- the original declared name and
the re-exported aliased name.

---------

Co-authored-by: Stefan J. Wernli <[email protected]>
  • Loading branch information
sezna and swernli authored Jul 16, 2024
1 parent 1deae14 commit 470c99a
Show file tree
Hide file tree
Showing 48 changed files with 1,773 additions and 423 deletions.
2 changes: 1 addition & 1 deletion compiler/qsc/benches/rca.rs
Original file line number Diff line number Diff line change
Expand Up @@ -151,7 +151,7 @@ fn lower_hir_package_store(hir_package_store: &HirPackageStore) -> PackageStore
for (id, unit) in hir_package_store {
fir_store.insert(
map_hir_package_to_fir(id),
Lowerer::new().lower_package(&unit.package),
Lowerer::new().lower_package(&unit.package, &fir_store),
);
}
fir_store
Expand Down
48 changes: 25 additions & 23 deletions compiler/qsc/src/interpret.rs
Original file line number Diff line number Diff line change
@@ -1,16 +1,15 @@
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT License.

mod debug;

#[cfg(test)]
mod tests;

mod circuit_tests;
mod debug;
#[cfg(test)]
mod debugger_tests;

#[cfg(test)]
mod circuit_tests;
mod package_tests;
#[cfg(test)]
mod tests;

use std::rc::Rc;

Expand Down Expand Up @@ -207,12 +206,10 @@ impl Interpreter {

let mut fir_store = fir::PackageStore::new();
for (id, unit) in compiler.package_store() {
fir_store.insert(
map_hir_package_to_fir(id),
qsc_lowerer::Lowerer::new()
.with_debug(dbg)
.lower_package(&unit.package),
);
let pkg = qsc_lowerer::Lowerer::new()
.with_debug(dbg)
.lower_package(&unit.package, &fir_store);
fir_store.insert(map_hir_package_to_fir(id), pkg);
}

let source_package_id = compiler.source_package_id();
Expand Down Expand Up @@ -265,10 +262,8 @@ impl Interpreter {
let mut fir_store = fir::PackageStore::new();
for (id, unit) in compiler.package_store() {
let mut lowerer = qsc_lowerer::Lowerer::new();
fir_store.insert(
map_hir_package_to_fir(id),
lowerer.lower_package(&unit.package),
);
let pkg = lowerer.lower_package(&unit.package, &fir_store);
fir_store.insert(map_hir_package_to_fir(id), pkg);
}

let source_package_id = compiler.source_package_id();
Expand Down Expand Up @@ -634,20 +629,27 @@ impl Interpreter {
if self.capabilities != TargetCapabilityFlags::all() {
return self.run_fir_passes(unit_addition);
}
let fir_package = self.fir_store.get_mut(self.package);
self.lowerer
.lower_and_update_package(fir_package, &unit_addition.hir);

self.lower_and_update_package(unit_addition);
Ok((self.lowerer.take_exec_graph(), None))
}

fn lower_and_update_package(&mut self, unit: &qsc_frontend::incremental::Increment) {
{
let fir_package = self.fir_store.get_mut(self.package);
self.lowerer
.lower_and_update_package(fir_package, &unit.hir);
}
let fir_package: &Package = self.fir_store.get(self.package);
qsc_fir::validate::validate(fir_package, &self.fir_store);
}

fn run_fir_passes(
&mut self,
unit: &qsc_frontend::incremental::Increment,
) -> std::result::Result<(Vec<ExecGraphNode>, Option<PackageStoreComputeProperties>), Vec<Error>>
{
let fir_package = self.fir_store.get_mut(self.package);
self.lowerer
.lower_and_update_package(fir_package, &unit.hir);
self.lower_and_update_package(unit);

let cap_results =
PassContext::run_fir_passes_on_fir(&self.fir_store, self.package, self.capabilities);
Expand Down Expand Up @@ -860,7 +862,7 @@ impl Debugger {
package,
self.position_encoding,
);
collector.visit_package(package);
collector.visit_package(package, &self.interpreter.fir_store);
let mut spans: Vec<_> = collector.statements.into_iter().collect();

// Sort by start position (line first, column next)
Expand Down
172 changes: 172 additions & 0 deletions compiler/qsc/src/interpret/package_tests.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,172 @@
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT License.

#![allow(clippy::needless_raw_string_hashes)]

use crate::{interpret::Interpreter, packages::BuildableProgram};
use indoc::indoc;
use qsc_data_structures::{language_features::LanguageFeatures, target::TargetCapabilityFlags};
use qsc_eval::output::CursorReceiver;
use qsc_frontend::compile::SourceMap;
use qsc_passes::PackageType;
use qsc_project::{PackageGraphSources, PackageInfo};
use rustc_hash::FxHashMap;

#[test]
fn import_and_call_reexport() {
let pkg_graph: PackageGraphSources = PackageGraphSources {
root: PackageInfo {
sources: vec![(
"PackageB.qs".into(),
indoc! {"
import Foo.DependencyA.Foo;
function Main() : Unit {
Foo([1, 2]);
Foo.DependencyA.MagicFunction();
}"}
.into(),
)],
language_features: LanguageFeatures::default(),
dependencies: [("Foo".into(), "PackageAKey".into())].into_iter().collect(),
package_type: None,
},
packages: [(
"PackageAKey".into(),
PackageInfo {
sources: vec![(
"Foo.qs".into(),
r#"
namespace DependencyA {
function MagicFunction() : Unit {
Message("hello from dependency A!");
}
export MagicFunction, Microsoft.Quantum.Core.Length as Foo;
}
"#
.into(),
)],
language_features: LanguageFeatures::default(),
dependencies: FxHashMap::default(),
package_type: None,
},
)]
.into_iter()
.collect(),
};

// This builds all the dependencies
let buildable_program = BuildableProgram::new(TargetCapabilityFlags::all(), pkg_graph);

assert!(
buildable_program.dependency_errors.is_empty(),
"dependencies should be built without errors"
);

let BuildableProgram {
store,
user_code,
user_code_dependencies,
..
} = buildable_program;

let user_code = SourceMap::new(user_code.sources, None);

let mut interpreter = Interpreter::new(
user_code,
PackageType::Exe,
TargetCapabilityFlags::all(),
LanguageFeatures::default(),
store,
&user_code_dependencies,
)
.expect("interpreter creation should succeed");

let mut cursor = std::io::Cursor::new(Vec::<u8>::new());
let mut receiver = CursorReceiver::new(&mut cursor);
let res = interpreter.eval_entry(&mut receiver);

assert!(res.is_ok(), "evaluation should succeed");

let output = String::from_utf8(cursor.into_inner()).expect("output should be valid utf-8");

assert_eq!(output, "hello from dependency A!\n");
}

#[test]
fn directly_call_reexport() {
let pkg_graph: PackageGraphSources = PackageGraphSources {
root: PackageInfo {
sources: vec![(
"PackageB.qs".into(),
indoc! {"
function Main() : Unit {
Foo.DependencyA.Foo([1, 2]);
Foo.DependencyA.MagicFunction();
}"}
.into(),
)],
language_features: LanguageFeatures::default(),
dependencies: [("Foo".into(), "PackageAKey".into())].into_iter().collect(),
package_type: None,
},
packages: [(
"PackageAKey".into(),
PackageInfo {
sources: vec![(
"Foo.qs".into(),
r#"
namespace DependencyA {
function MagicFunction() : Unit {
Message("hello from dependency A!");
}
export MagicFunction, Microsoft.Quantum.Core.Length as Foo;
}
"#
.into(),
)],
language_features: LanguageFeatures::default(),
dependencies: FxHashMap::default(),
package_type: None,
},
)]
.into_iter()
.collect(),
};

// This builds all the dependencies
let buildable_program = BuildableProgram::new(TargetCapabilityFlags::all(), pkg_graph);

assert!(
buildable_program.dependency_errors.is_empty(),
"dependencies should be built without errors"
);

let BuildableProgram {
store,
user_code,
user_code_dependencies,
..
} = buildable_program;

let user_code = SourceMap::new(user_code.sources, None);

let mut interpreter = Interpreter::new(
user_code,
PackageType::Exe,
TargetCapabilityFlags::all(),
LanguageFeatures::default(),
store,
&user_code_dependencies,
)
.expect("interpreter creation should succeed");

let mut cursor = std::io::Cursor::new(Vec::<u8>::new());
let mut receiver = CursorReceiver::new(&mut cursor);
let res = interpreter.eval_entry(&mut receiver);

assert!(res.is_ok(), "evaluation should succeed");

let output = String::from_utf8(cursor.into_inner()).expect("output should be valid utf-8");

assert_eq!(output, "hello from dependency A!\n");
}
2 changes: 1 addition & 1 deletion compiler/qsc_ast/src/ast.rs
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ fn set_indentation<'a, 'b>(
0 => indent.with_str(""),
1 => indent.with_str(" "),
2 => indent.with_str(" "),
_ => unimplemented!("intentation level not supported"),
_ => unimplemented!("indentation level not supported"),
}
}

Expand Down
2 changes: 1 addition & 1 deletion compiler/qsc_codegen/src/qir.rs
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ use qsc_rir::{
fn lower_store(package_store: &qsc_frontend::compile::PackageStore) -> qsc_fir::fir::PackageStore {
let mut fir_store = qsc_fir::fir::PackageStore::new();
for (id, unit) in package_store {
let package = qsc_lowerer::Lowerer::new().lower_package(&unit.package);
let package = qsc_lowerer::Lowerer::new().lower_package(&unit.package, &fir_store);
fir_store.insert(map_hir_package_to_fir(id), package);
}
fir_store
Expand Down
74 changes: 71 additions & 3 deletions compiler/qsc_data_structures/src/namespaces.rs
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
mod tests;

use rustc_hash::{FxHashMap, FxHashSet};
use std::{cell::RefCell, fmt::Display, iter::Peekable, ops::Deref, rc::Rc};
use std::{cell::RefCell, collections::BTreeMap, fmt::Display, iter::Peekable, ops::Deref, rc::Rc};

pub const PRELUDE: [[&str; 3]; 4] = [
["Microsoft", "Quantum", "Canon"],
Expand All @@ -15,7 +15,7 @@ pub const PRELUDE: [[&str; 3]; 4] = [
];

/// An ID that corresponds to a namespace in the global scope.
#[derive(Clone, Copy, Debug, Eq, Hash, PartialEq, Default)]
#[derive(Clone, Copy, Debug, Eq, Hash, PartialEq, Default, PartialOrd, Ord)]
pub struct NamespaceId(usize);
impl NamespaceId {
/// Create a new namespace ID.
Expand Down Expand Up @@ -196,7 +196,75 @@ impl NamespaceTreeRoot {
.borrow_mut()
.insert_or_find_namespace(ns.into_iter().peekable(), &mut self.assigner);

id.expect("empty name should not be passed into namespace insertion")
id.expect("empty name checked for above")
}

pub fn insert_or_find_namespace_from_root_with_id(
&mut self,
mut ns: Vec<Rc<str>>,
root: NamespaceId,
base_id: NamespaceId,
) {
if ns.is_empty() {
return;
}
let (_root_name, root_contents) = self.find_namespace_by_id(&root);
// split `ns` into [0..len - 1] and [len - 1]
let suffix = ns.split_off(ns.len() - 1)[0].clone();
let prefix = ns;

// if the prefix is empty, we are inserting into the root
if prefix.is_empty() {
self.insert_with_id(Some(root), base_id, &suffix);
} else {
let prefix_id = root_contents
.borrow_mut()
.insert_or_find_namespace(prefix.into_iter().peekable(), &mut self.assigner)
.expect("empty name checked for above");

self.insert_with_id(Some(prefix_id), base_id, &suffix);
}
}

/// Each item in this iterator is the same, single namespace. The reason there are multiple paths for it,
/// each represented by a `Vec<Rc<str>>`, is because there may be multiple paths to the same
/// namespace, through aliasing or re-exports.
pub fn iter(&self) -> std::collections::btree_map::IntoValues<NamespaceId, Vec<Vec<Rc<str>>>> {
let mut stack = vec![(vec![], self.tree.clone())];
let mut result: Vec<(NamespaceId, Vec<Rc<str>>)> = vec![];
while let Some((names, node)) = stack.pop() {
result.push((node.borrow().id, names.clone()));
for (name, child) in node.borrow().children() {
let mut new_names = names.clone();
new_names.push(name.clone());
stack.push((new_names, child.clone()));
}
if node.borrow().children().is_empty() {
result.push((node.borrow().id, names));
}
}

// flatten the result into a list of paths

// use a btree map here instead of a hash map for deterministic iteration --
// while it shouldn't be consequential, any nondeterminism in a compiler makes
// things more difficult to track down then they go wrong.
let mut flattened_result = BTreeMap::default();
for (id, names) in result {
let entry = flattened_result.entry(id).or_insert_with(Vec::new);
entry.push(names);
}

flattened_result.into_values()
}
}

impl IntoIterator for &NamespaceTreeRoot {
type Item = Vec<Vec<Rc<str>>>;
type IntoIter = std::collections::btree_map::IntoValues<NamespaceId, Vec<Vec<Rc<str>>>>;

fn into_iter(self) -> Self::IntoIter {
self.iter()
}
}

Expand Down
Loading

0 comments on commit 470c99a

Please sign in to comment.