-
Notifications
You must be signed in to change notification settings - Fork 24
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[LLT-5831] QNAP: move cgi logic to teliod #1012
base: main
Are you sure you want to change the base?
Conversation
4d75c96
to
03917df
Compare
03917df
to
38938fe
Compare
Can we use instead something like https://docs.rs/cgi2/latest/cgi/ or https://docs.rs/cgi/0.6.0/cgi/ , so that we avoid implementing irrelevant details of cgi/http, and focus on our business logic? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
imho looks strange, like someone wanted to write a shell script, but had to do it in rust for some reason
clis/teliod/src/qnap.rs
Outdated
enum Action { | ||
Start, | ||
Stop, | ||
UpdateConfig, | ||
GetStatus, | ||
GetTeliodLogs, | ||
GetMeshnetLogs, | ||
Invalid, | ||
} | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What is the reason of creating this intermediate enum? It looks to me that you just make two match
statements instead of one, without any visible gains.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It looks like that indeed, however I thought it would be good to have it like this so that we can expand it in the future.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
in the end I'll change it with the http
crate objects as you suggested below
clis/teliod/src/qnap.rs
Outdated
} | ||
|
||
pub fn teliod_cfg() -> String { | ||
format!("{}/teliod.cfg", Self::QPKG_DIR) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Teliod knowing where is the config file is a bit weird idea imho, in such case I would refactor it to exclude that argument and hardcode it in the rest of the code as well.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
sorry I didn't understand, can you explain?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Currently teliod daemon starts with config file path provided as an argument - the situation where this path is at the same time hardcoded provided to and hardcoded in teliod doesn't feel right.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Other platforms certainly have a different path for the config file whereas for QNAP this is a static path, hence the idea of having it hardcoded.
@stalowyjez that was more or less what happened, but what do you think it looks strange besides other comments? |
@tomaszklak thanks for the recommendations, will take a look! |
I mean it just doesn't look like a right tool for the job to me |
clis/teliod/src/qnap.rs
Outdated
} | ||
|
||
pub fn teliod_log() -> String { | ||
"/var/log/teliod.log".to_owned() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Shouldn't these be retrieved from the config?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's one possibility. Do you think other file paths should also be set on the config? or just teliod.log
?
38938fe
to
0abc522
Compare
0abc522
to
687836b
Compare
687836b
to
a21cb40
Compare
a21cb40
to
4cd24a8
Compare
4cd24a8
to
2608ac7
Compare
2608ac7
to
5f8d630
Compare
5f8d630
to
2657c32
Compare
(&Method::POST, _) => start_daemon(), | ||
(&Method::DELETE, _) => stop_daemon(), | ||
(&Method::PATCH, _) => { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please add verification of path.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
verification of path?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For example in the documentation we say:
- **Endpoint**: `/`
- **Method**: `POST`
But in code we start daemon after any post, regardless of the path. We should verify that the path is exacly /
. Similar for DELETE
and PATCH
methods.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
the Endpoint path /
is relative to the teliod.cgi path. So /
is equivalent to /cgi-bin/qpkg/teliod.cgi
. I can check against this hardcoded path..
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is user able to control the path parameter?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If we expect that it will always be the same value (/cgi-bin/qpkg/teliod.cgi
?) than there should be no reason not to check - if we are mistaken we want to know about this.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
fair enough, added
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ugh.... I meant query
not path
🤦
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hey let's not do path verification in this PR, let's leave it after we have a web page logic
also path() should only contain string afeter cgi-bin/qpkg/teliod.cgi,soforexample if cgi-bin/qpkg/teliod.cgi/xxx/x would be request, path should be /xxx/x.
In my setup i will rely that '/' path is return index page, and '/style.css' will return css and so on, so let's leave path out of this pr.
Query tipycaly should not be fragile, we disccuseed with luis already, but proper parsing should be done so for example:
?some-id=1&some-other=2 should be equivalent to ?some-other=2&some-id=1
We will add this with auth logic
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
will remove path verification. Query will indeed be validated on a next PR
clis/teliod/src/qnap.rs
Outdated
assert_eq!( | ||
updated_config.log_level, | ||
tracing::level_filters::LevelFilter::INFO | ||
); | ||
assert_eq!(updated_config.log_file_path, "/new/path/to/log"); | ||
assert_eq!( | ||
updated_config.authentication_token, | ||
"bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb" | ||
); | ||
assert_eq!( | ||
updated_config.app_user_uid, | ||
Uuid::parse_str("11111111-1111-1111-1111-111111111111").unwrap() | ||
); | ||
assert_eq!(updated_config.interface.name, "eth1"); | ||
assert_eq!( | ||
updated_config.interface.config_provider, | ||
InterfaceConfigurationProvider::Ifconfig | ||
); | ||
assert_eq!(updated_config.http_certificate_file_path, None); | ||
let mqtt_default_cfg = MqttConfig::default(); | ||
assert_eq!( | ||
updated_config.mqtt.backoff_initial, | ||
mqtt_default_cfg.backoff_initial | ||
); | ||
assert_eq!( | ||
updated_config.mqtt.backoff_maximal, | ||
mqtt_default_cfg.backoff_maximal | ||
); | ||
assert_eq!( | ||
updated_config.mqtt.reconnect_after_expiry, | ||
mqtt_default_cfg.reconnect_after_expiry | ||
); | ||
assert_eq!( | ||
updated_config.mqtt.certificate_file_path, | ||
mqtt_default_cfg.certificate_file_path | ||
); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: I think it would be more readable if you would create the expected struct and compare it against the updated_config
, instead of manually doing it field by field.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Agreed, changed
clis/teliod/src/qnap.rs
Outdated
|
||
let update_body = r#" | ||
{ | ||
"authentication_token": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please add tests for all the fields.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
wdym? Add tests for each field partial update?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
or changing all fields in a single test is enough?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The important part is that the whole production code is covered - whether it's in one test or many is less important and I will leave this up to you to judge which is better in this case.
f28351d
to
e88801f
Compare
clis/teliod/Cargo.toml
Outdated
@@ -34,6 +34,13 @@ anyhow.workspace = true | |||
smart-default = "0.7.1" | |||
base64 = "0.22.1" | |||
dirs = "4.0.0" | |||
const_format = { version = "0.2.33", optional = true } | |||
rust-cgi = { version = "0.7.1", optional = true } | |||
arrayvec = "0.7.6" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This doesn't seem to be used anywhere - please remove it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
removed
(&Method::POST, _) => start_daemon(), | ||
(&Method::DELETE, _) => stop_daemon(), | ||
(&Method::PATCH, _) => { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ugh.... I meant query
not path
🤦
clis/teliod/src/qnap.rs
Outdated
fn kill_teliod_process() -> Result<(), io::Error> { | ||
match fs::read_to_string(PID_FILE) { | ||
Ok(pid) => { | ||
let _ = Command::new("kill").arg("-9").arg(pid.trim()).status(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, I'm thinking that maybe we shouldn't be calling kill
at all - it might turn out it's possible somehow to trick our cgi handler to kill arbitrary processes on qnap.
clis/teliod/src/qnap.rs
Outdated
} else { | ||
match kill_teliod_process() { | ||
Ok(_) => text_response(StatusCode::OK, "Application stopped successfully."), | ||
_ => text_response(StatusCode::GONE, "Application not found."), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Might be good to log the error somewhere, or include it in the response.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
kill_teliod_process
function removed
clis/teliod/src/qnap.rs
Outdated
text_response(StatusCode::OK, "Application stopped successfully.") | ||
} else { | ||
match kill_teliod_process() { | ||
Ok(_) => text_response(StatusCode::OK, "Application stopped successfully."), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it would be best not to use the same exact message here and in the ok case.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
kill_teliod_process
function removed
clis/teliod/src/qnap.rs
Outdated
fn stop_daemon() -> Response { | ||
if shutdown_teliod().is_ok() { | ||
text_response(StatusCode::OK, "Application stopped successfully.") | ||
} else { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's not ignore this error and include it in below messages or log it somehow.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added shutdown error log
clis/teliod/src/qnap.rs
Outdated
|
||
#[test] | ||
fn test_update_config() { | ||
let _lock = TEST_SEQ_MUTEX.lock().unwrap(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This can be replaced with serial_test
, which we already use across other crates in libtelio.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
configure_interface::InterfaceConfigurationProvider, | ||
}; | ||
|
||
pub const TELIOD_CFG: &str = "/tmp/teliod_config.json"; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this will fail on windows - can you change it to use https://crates.io/crates/tempfile ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this is out of scope for now, we can make this as an issue, to make cgi standalone and cross platform
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is not about making cgi cross platform - it's about making unit tests pass for developer(s?) on windows.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How would windows devs run these unit tests?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hm, you right - there is no configurability in this module and all paths are completely hardcoded. It's probably relatively cheap to make paths be configurable with a prefix (which would be empty in production) and set it to temporary dir in tests. But fell free to treat this thread as nitpick and ignore it if you want.
e88801f
to
97e2acd
Compare
Original work done in #1012. This is restructure and reabase on common base.
Original work done in #1012. This is restructure and reabase on common base.
97e2acd
to
a696f1e
Compare
@@ -76,6 +76,19 @@ impl CommandListener { | |||
}) | |||
.await | |||
} | |||
ClientCmd::QuitDaemon => |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
IMHO it would be nice to log commands executed (e.g. debug!()
or maybe info!()
). This helps puts things into perspective when debugging various issues.
Obviously - we should log all commands not just QuitDaemon
. Although it does not have to be this PR, it may be separate one
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Every command received by the daemon is logged here.
#[serde(rename_all = "lowercase")] | ||
pub enum InterfaceConfigurationProvider { | ||
#[default] | ||
Manual, | ||
Ifconfig, | ||
Iproute, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is just an idea, so you may resolve this message once you have read it.
Eventually it would be really nice to use netlink for network configs instead of spawning shell commands :).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think is a good idea but out of scope of this PR :)
|
||
fn deserialize_partial_authentication_token<'de, D: Deserializer<'de>>( | ||
deserializer: D, | ||
) -> Result<Option<String>, D::Error> { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe it would be worth creating dedicated type for Authentication Token? This would allow us to do some type-safe things (e.g. derive zeroize
and similar). Restrict printing/logging and similar. Reliably enforce invariants (e.g. hexdigits only and 64 lengh max)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That was my initial thought, I've changed my mind though when I realized the effort and started to go out-of-scope of the PR. This is definitely a TODO for the next PRs.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
To make sure that feature qnap
does not accidentally leak out of the feature, would it make sense to mark all of this file as #[cfg(feature = "qnap")]
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Didn't fully understand what you mean with "leak out of the future". Anyways, this tree structure will change in the next PR so I think is trivial
clis/teliod/src/main.rs
Outdated
@@ -96,13 +105,21 @@ async fn main() -> Result<(), TeliodError> { | |||
if DaemonSocket::get_ipc_socket_path()?.exists() { | |||
Err(TeliodError::DaemonIsRunning) | |||
} else { | |||
#[cfg(feature = "qnap")] | |||
let _pid_guard = qnap::PidFile::new().await?; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I wonder do we have a need for pidfiles?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That's the pattern used for QNAP applications, for start/stop routines (eg: qnap/shared/NordSecMeshnet.sh).
good point, initially we were killing the process from the cgi script, which was in the meantime removed and we don't need the Pidfile now.. so will remove Pidfile from the app side.
debug!("Overriding token from env"); | ||
config.authentication_token = t; | ||
if token.len() == 64 && token.chars().all(|c| c.is_ascii_hexdigit()) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It would be nice to make AuthorizationToken type which would enforce these invariants. e.g. something on the lines like:
struct AuthorizationToken {
...
};
impl AuthorizationToken {
pub fn new(String token) -> Result<Self, Error> {
if token.len() != 64 || token.chars().all(|c| c.is_ascii_hexdigit()) {
return Err(...);
}
...
}
}
And then every place, which needs access to AuthorizationToken would use AuthorizationToken
struct instead of String. The benefit of such approach is that we can be sure (in a type safe manner) that invariants (e.g. length and symbol requirements) are always enforced for such type and makes it significantly harder to forget validation by accident. What is more, there it reduced code repetition, as even this length and hexdigit requirement had to be replicated a few times in current PR.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I had pretty much the same thought here: https://github.com/NordSecurity/libtelio/pull/1012/files#r1894030576
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Agreed! However this will be part of the next PR.
const QPKG_DIR: &str = "/share/CACHEDEV1_DATA/.qpkg/NordSecurityMeshnet"; | ||
const TELIOD_BIN: &str = concatcp!(QPKG_DIR, "/teliod"); | ||
const MESHNET_LOG: &str = concatcp!(QPKG_DIR, "/meshnet.log"); | ||
const TELIOD_LOG: &str = "/var/log/teliod.log"; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think this should be called "teliod log", as teliod logs with meshnet logger
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hm.. what do you mean with "teliod logs with meshnet logger"?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I mean it uses tracing logs as Telio, so all of them go to the same file. Here there is the initialization for both meshnet and teliod:
libtelio/clis/teliod/src/daemon.rs
Line 184 in 26fe43a
tracing_subscriber::fmt() |
it doesn't really use stdout for logging (or it shouldn't at least)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We have meshnet logs (from libtelio) which use the tracing crate and then we have teliod logs which log the stdout/stderr of the teliod process.
The link you shared has tracing object initlization for meshnet logs, however you say both meshnet and teliod? I don't understand
) | ||
} | ||
}; | ||
match Command::new("setsid") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
imho it would be nicer to fork than using cli
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
fork()
is still used, however when used alone, the child process inherits the parent session id which makes the process quit after the parent session ends. Instead setsid
sets a new session id for the child process.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I meant sth like we had in Tcli, it worked well
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you be more specific pls?
} | ||
} | ||
|
||
fn get_meshnet_logs() -> Response { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is there a good reason to separate teliod
and meshnet
logs? maybe it would be worth to interleave them, like apps do? In that case it is slightly easier to get them from users, as there is single file instead of two.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In my POV I'd prefer to keep them separated simply because meshnet logs will bury teliod
logs (too many meshnet logs) which also convey a different category of information.
a696f1e
to
e58d335
Compare
e58d335
to
bf4f507
Compare
Problem
CGI consists in a bash script that handles http requests and directly control teliod daemon execution.
As in the near future more features (eg: authentication) are to be added, after internal discussions it was decided that would be more secure and easy to maintain/implement if the handling of the requests is moved to the teliod binary.
Solution
Make the CGI script redirect http requests to teliod.
☑️ Definition of Done checklist