Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Split the start state more conservatively when a parser contains decl initializers #4902

Merged
merged 6 commits into from
Sep 12, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
96 changes: 72 additions & 24 deletions frontends/p4/moveDeclarations.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -97,7 +97,30 @@ const IR::Node *MoveDeclarations::postorder(IR::Declaration_Constant *decl) {

////////////////////////////////////////////////////////////////////

// Returns true iff the given state is _directly_ reachable from any of the given
// set of states.
bool isReachable(cstring stateName, const IR::IndexedVector<IR::ParserState> &states) {
for (const auto *state : states) {
const auto *selectExpr = state->selectExpression;
if (selectExpr == nullptr) continue; // implicit reject transition

if (const auto *sel = selectExpr->to<IR::SelectExpression>()) {
// select expression
for (const auto *selectCase : sel->selectCases)
if (selectCase->state->path->name == stateName) return true;
} else if (const auto *pathExpr = selectExpr->to<IR::PathExpression>()) {
// direct transition
if (pathExpr->path->name == stateName) return true;
}
}

return false;
}

const IR::Node *MoveInitializers::preorder(IR::P4Parser *parser) {
oldStart = nullptr;
loopsBackToStart = false;

// Determine if we have anything to do in this parser.
// Check if we have top-level declarations with intializers.
auto someInitializers = false;
Expand All @@ -109,24 +132,22 @@ const IR::Node *MoveInitializers::preorder(IR::P4Parser *parser) {
}
}
}

if (someInitializers) {
// Check whether any of the parser's states loop back to the start state.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't there a class which builds the reachability graph for parser states?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure, but that might help. @fruffy @vlstill ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

./frontends/p4/parserCallGraph.h

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CallGraph::reachable(start, reachable) will always include start in the set of reachable states, so this will not really help simplify things.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you can check the set of in-edge of the start state

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

but frankly it would be useful to understand why the results are wrong. If there is a bug in the callgraph, it would be nice to know that. If the meaning of the isCalle() function is different than expected, it should be documented. If the rule for splitting is wrong, it would be good to know that too.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mihaibudiu Personally, I find it easier to initially analyze changes by looking at git diffs, which is why I shared the link to the diff with the C++ changes as well as corresponding updated tests outputs. If you prefer to initially analyze changes by checking out the changes locally, running the tests, and inspecting the tests failures locally, you can easily do this by recreating the changes minus tests outputs updates by fetching my fork and using git checkout commands.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

but frankly it would be useful to understand why the results are wrong. If there is a bug in the callgraph, it would be nice to know that. If the meaning of the isCalle() function is different than expected, it should be documented. If the rule for splitting is wrong, it would be good to know that too.

If there's a problem with the ParserCallGraph, that is outside the scope of this PR, and I'm not volunteering to spend time debugging this. But others may feel free to do so :)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am no longer working on this project, but I will try to find time to fix bugs that I am responsible for.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mihaibudiu For example: 4b0ec77#diff-cd2b40893d904a1015f5d71d39014e909e148f5db324e0ebb77484e99c3cc6c7R2-R7

Split is expected in this case because start is reachable from other states. So either I'm using ParserCallGraph incorrectly, or it's not working properly.

I am no longer working on this project, but I will try to find time to fix bugs that I am responsible for.

I did not realize you were no longer working on this project. Please do not feel obligated to fix even the bugs you are responsible for. But any help is of course appreciated :)

// If this is the case, then the parser's decl initializers cannot be moved
// to the start state. A new start state that is never looped back to must
// be inserted at the head of the graph.
loopsBackToStart = isReachable(IR::ParserState::start, parser->states);

newStartName = nameGen.newName(IR::ParserState::start.string_view());
oldStart = parser->states.getDeclaration(IR::ParserState::start)->to<IR::ParserState>();
CHECK_NULL(oldStart);
}
return parser;
}

const IR::Node *MoveInitializers::postorder(IR::P4Parser *parser) {
if (oldStart == nullptr) return parser;
auto newStart = new IR::ParserState(IR::ID(IR::ParserState::start), *toMove,
new IR::PathExpression(newStartName));
toMove = new IR::IndexedVector<IR::StatOrDecl>();
parser->states.insert(parser->states.begin(), newStart);
return parser;
}

const IR::Node *MoveInitializers::postorder(IR::Declaration_Variable *decl) {
const IR::Node *MoveInitializers::preorder(IR::Declaration_Variable *decl) {
if (getContext() == nullptr) return decl;
auto parent = getContext()->node;
if (!parent->is<IR::P4Control>() && !parent->is<IR::P4Parser>())
Expand All @@ -136,34 +157,61 @@ const IR::Node *MoveInitializers::postorder(IR::Declaration_Variable *decl) {

auto varRef = new IR::PathExpression(decl->name);
auto assign = new IR::AssignmentStatement(decl->srcInfo, varRef, decl->initializer);
toMove->push_back(assign);
toMove.push_back(assign);
decl->initializer = nullptr;
return decl;
}

const IR::Node *MoveInitializers::postorder(IR::ParserState *state) {
if (oldStart == nullptr) return state;
if (state->name == IR::ParserState::start) state->name = newStartName;
return state;
}
if (state->name == IR::ParserState::start && !toMove.empty()) {
if (!loopsBackToStart) {
// Just prepend all initializing assignments to the original start
// state if there are no loops back to the start state.
state->components.insert(state->components.begin(), toMove.begin(), toMove.end());
toMove.clear();
} else {
// Otherwise, we need to insert all such assignments into a new start state
// that can only run once, so that the assignments are only executed one time.
state->name = newStartName;
}
}

const IR::Node *MoveInitializers::postorder(IR::P4Control *control) {
if (toMove->empty()) return control;
toMove->append(control->body->components);
auto newBody =
new IR::BlockStatement(control->body->srcInfo, control->body->annotations, *toMove);
control->body = newBody;
toMove = new IR::IndexedVector<IR::StatOrDecl>();
return control;
return state;
}

const IR::Node *MoveInitializers::postorder(IR::Path *path) {
if (oldStart == nullptr || path->name != IR::ParserState::start) return path;
if (!oldStart || !loopsBackToStart || path->name != IR::ParserState::start) return path;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if (!oldStart || !loopsBackToStart || path->name != IR::ParserState::start) return path;
if (oldStart == nullptr || !loopsBackToStart || path->name != IR::ParserState::start) return path;


// Only rename start state references if the parser contains initializing assignments
// and loops back to the start state.
auto decl = getDeclaration(getOriginal()->to<IR::Path>());
if (!decl->is<IR::ParserState>()) return path;
return new IR::Path(newStartName);
}

const IR::Node *MoveInitializers::postorder(IR::P4Parser *parser) {
if (oldStart && loopsBackToStart) {
// Only add a new state if the parser has initializers and contains one
// or more states that loop back to the start state.
auto newStart = new IR::ParserState(IR::ID(IR::ParserState::start), toMove,
new IR::PathExpression(newStartName));
toMove.clear();
parser->states.insert(parser->states.begin(), newStart);
}

return parser;
}

const IR::Node *MoveInitializers::postorder(IR::P4Control *control) {
if (toMove.empty()) return control;
toMove.append(control->body->components);
auto newBody =
new IR::BlockStatement(control->body->srcInfo, control->body->annotations, toMove);
control->body = newBody;
toMove.clear();
return control;
}

Visitor::profile_t MoveInitializers::init_apply(const IR::Node *node) {
auto rv = Transform::init_apply(node);
node->apply(nameGen);
Expand Down
16 changes: 7 additions & 9 deletions frontends/p4/moveDeclarations.h
Original file line number Diff line number Diff line change
Expand Up @@ -101,22 +101,20 @@ class MoveDeclarations : public Transform {
*/
class MoveInitializers : public Transform, public ResolutionContext {
MinimalNameGenerator nameGen;
IR::IndexedVector<IR::StatOrDecl> *toMove; // This contains just IR::AssignmentStatement
IR::IndexedVector<IR::StatOrDecl> toMove; // This contains just IR::AssignmentStatement
const IR::ParserState *oldStart; // nullptr if we do not want to rename the start state
cstring newStartName; // name allocated to the old start state
cstring newStartName; // name allocated to the new start state
bool loopsBackToStart; // true if start is reachable from any other state

public:
MoveInitializers()
: toMove(new IR::IndexedVector<IR::StatOrDecl>()), oldStart(nullptr), newStartName("") {
setName("MoveInitializers");
}
MoveInitializers() : newStartName("") { setName("MoveInitializers"); }
profile_t init_apply(const IR::Node *node) override;
const IR::Node *preorder(IR::P4Parser *parser) override;
const IR::Node *postorder(IR::P4Parser *parser) override;
const IR::Node *postorder(IR::Declaration_Variable *decl) override;
const IR::Node *preorder(IR::Declaration_Variable *decl) override;
const IR::Node *postorder(IR::ParserState *state) override;
const IR::Node *postorder(IR::P4Control *control) override;
const IR::Node *postorder(IR::Path *path) override;
const IR::Node *postorder(IR::P4Parser *parser) override;
const IR::Node *postorder(IR::P4Control *control) override;
};

} // namespace P4
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
parser Parser1() {
bit<8> l = 0;
state start {
transition next;
}
state next {
transition select(l) {
default: accept;
}
}
}

parser Parser2() {
state start {
transition accept;
}
}

parser proto();
package top(proto p1, proto p2);
top(Parser1(), Parser2()) main;

Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
parser Parser1() {
bit<8> l = 0;
state start {
transition next;
}
state next {
transition select(l) {
2 : start;
default: accept;
}
}
}

parser Parser2() {
state start {
transition accept;
}
}
Comment on lines +14 to +18
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd add some initializer here just to see that there is no confusion between the two parsers.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd prefer to not change this test (it is the case taken from #4901) but I added a new one.


parser proto();
package top(proto p1, proto p2);
top(Parser1(), Parser2()) main;

Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
parser Parser1() {
bit<8> l = 0;
state start {
transition next;
}
state next {
transition select(l) {
2 : start;
default: accept;
}
}
}

parser Parser2() {
bit<7> l = 1;
state start {
transition select (l) {
0 : s0;
default : accept;
}
}
state s0 {
transition accept;
}
}

parser proto();
package top(proto p1, proto p2);
top(Parser1(), Parser2()) main;

3 changes: 0 additions & 3 deletions testdata/p4_16_samples_outputs/default-frontend.p4
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,6 @@ parser p0(packet_in p, out Header h) {
@name("p0.b") bool b_0;
state start {
b_0 = true;
transition start_0;
}
state start_0 {
p.extract<Header>(h);
transition select(h.data, b_0) {
(default, true): next;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -214,9 +214,6 @@ parser FabricParser(packet_in packet, out parsed_headers_t hdr, inout fabric_met
@name("FabricParser.tmp") bit<4> tmp;
@name("FabricParser.tmp_0") bit<4> tmp_0;
state start {
transition start_0;
}
state start_0 {
transition select(standard_metadata.ingress_port) {
9w255: parse_packet_out;
default: parse_ethernet;
Expand Down
3 changes: 0 additions & 3 deletions testdata/p4_16_samples_outputs/initializers-frontend.p4
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,6 @@ parser P() {
@name("P.fake") Fake() fake_0;
state start {
x_0 = 32w0;
transition start_0;
}
state start_0 {
fake_0.call(x_0);
transition accept;
}
Expand Down
3 changes: 0 additions & 3 deletions testdata/p4_16_samples_outputs/issue1538-frontend.p4
Original file line number Diff line number Diff line change
Expand Up @@ -26,9 +26,6 @@ parser ParserImpl(packet_in packet, out headers hdr, inout metadata meta, inout
x = (bit<16>)standard_metadata.ingress_port;
retval = x + 16w1;
tmp_port_0 = retval;
transition start_0;
}
state start_0 {
packet.extract<ethernet_t>(hdr.ethernet);
x_6 = hdr.ethernet.etherType;
retval_0 = x_6 + 16w1;
Expand Down
3 changes: 0 additions & 3 deletions testdata/p4_16_samples_outputs/issue2314-frontend.p4
Original file line number Diff line number Diff line change
Expand Up @@ -28,9 +28,6 @@ struct m {
parser MyParser(packet_in b, out h hdr, inout m meta, inout standard_metadata_t std) {
@name("MyParser.l3.etherType") bit<16> l3_etherType;
state start {
transition start_0;
}
state start_0 {
b.extract<ethernet_t>(hdr.ether);
transition L3_start;
}
Expand Down
3 changes: 0 additions & 3 deletions testdata/p4_16_samples_outputs/issue281-frontend.p4
Original file line number Diff line number Diff line change
Expand Up @@ -42,9 +42,6 @@ struct m {
parser MyParser(packet_in b, out h hdr, inout m meta, inout standard_metadata_t std) {
@name("MyParser.l3.etherType") bit<16> l3_etherType;
state start {
transition start_0;
}
state start_0 {
hdr.ether.setInvalid();
hdr.vlan.setInvalid();
hdr.ipv4.setInvalid();
Expand Down
3 changes: 0 additions & 3 deletions testdata/p4_16_samples_outputs/issue2957-frontend.p4
Original file line number Diff line number Diff line change
Expand Up @@ -39,9 +39,6 @@ parser p(packet_in pkt, out Headers hdr) {
tmp_3 = hdr.h[tmp_2];
tmp_4 = extern_call(tmp_3);
hdr.h[tmp_2] = tmp_3;
transition start_0;
}
state start_0 {
pkt.extract<ethernet_t>(hdr.eth_hdr);
pkt.extract<H>(hdr.h.next);
transition accept;
Expand Down
3 changes: 0 additions & 3 deletions testdata/p4_16_samples_outputs/issue361-bmv2-frontend.p4
Original file line number Diff line number Diff line change
Expand Up @@ -17,9 +17,6 @@ parser MyParser(packet_in b, out my_packet p, inout my_metadata m, inout standar
@name("MyParser.bv") bool bv_0;
state start {
bv_0 = true;
transition start_0;
}
state start_0 {
transition select(bv_0) {
false: next;
true: accept;
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
parser Parser1() {
bit<8> l = 8w0;
state start {
transition next;
}
state next {
transition select(l) {
default: accept;
}
}
}

parser Parser2() {
state start {
transition accept;
}
}

parser proto();
package top(proto p1, proto p2);
top(Parser1(), Parser2()) main;
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
parser Parser1() {
@name("Parser1.l") bit<8> l_0;
state start {
l_0 = 8w0;
transition select(l_0) {
default: accept;
}
}
}

parser Parser2() {
state start {
transition accept;
}
}

parser proto();
package top(proto p1, proto p2);
top(Parser1(), Parser2()) main;
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
parser Parser1() {
state start {
transition accept;
}
}

parser Parser2() {
state start {
transition accept;
}
}

parser proto();
package top(proto p1, proto p2);
top(Parser1(), Parser2()) main;
Loading
Loading