From 024ec706edc08c5b5caed0374811d0e7a8e32fa4 Mon Sep 17 00:00:00 2001 From: jamesroutley Date: Thu, 26 Dec 2024 18:22:39 +0000 Subject: [PATCH] Build website (automatic) --- docs/index.html | 54 +- docs/log.txt | 86 ++-- .../a-minecraft-server-written-in-cobol.html | 18 +- .../a-new-learning-experience-on-mdn.html | 53 ++ ...rch-idea-false-statements-in-e-graphs.html | 2 +- .../posts/cognitive-load-is-what-matters.html | 305 ++++++++++++ ...oks-better-than-black-frame-insertion.html | 63 --- ...-general-purpose-programming-language.html | 12 +- ...to-build-an-electrically-heated-table.html | 441 ++++++++--------- docs/posts/into-cps-never-to-return.html | 4 +- ...-valid-service-on-all-65535-tcp-ports.html | 274 ++--------- ...server-sent-events-sse-are-underrated.html | 312 ------------ ...ersonal-knowledge-management-software.html | 460 ++++++++++++++++-- ...he-frontline-of-a-possible-hybrid-war.html | 2 +- ...fter-516-days-7-700-miles-on-the-road.html | 2 +- docs/posts/weeknotes-28.html | 2 +- 16 files changed, 1117 insertions(+), 973 deletions(-) create mode 100644 docs/posts/a-new-learning-experience-on-mdn.html create mode 100644 docs/posts/cognitive-load-is-what-matters.html delete mode 100644 docs/posts/crt-simulation-in-a-gpu-shader-looks-better-than-black-frame-insertion.html delete mode 100644 docs/posts/server-sent-events-sse-are-underrated.html diff --git a/docs/index.html b/docs/index.html index c9e5cf57327..2a92a843f22 100644 --- a/docs/index.html +++ b/docs/index.html @@ -16,6 +16,15 @@

News

+
  • + Software for stationery lovers (vrklovespaper.substack.com) +
    • + + + + + +
  • Show HN: A singing synthesizer for the browser with automatic 3-part harmony (pbat.ch)
    • @@ -89,15 +98,6 @@

    News

    -
  • - Server-Sent Events (SSE) Are Underrated (igorstechnoclub.com) -
    • - - - - - -
  • I thought I found a bug (www.os2museum.com)
    • @@ -161,15 +161,6 @@

    News

    -
  • - CRT Simulation in a GPU Shader, Looks Better Than Black Frame Insertion (blurbusters.com) -
    • - - - - - -
  • VPR: Nordic's First RISC-V Processor (danielmangum.com)
    • @@ -233,6 +224,24 @@

    News

    +
  • + A new learning experience on MDN (developer.mozilla.org) +
    • + + + + + +
  • + Show HN: Super Snowflake Maker (supersnowflakemaker.com) +
    • + + + + + + +
  • Feel, don't think (ntietz.com)
    • @@ -242,6 +251,15 @@

    News

    +
  • + Cognitive load is what matters (minds.md) +
    • + + + + + +
  • AI-generated tools can make programming more fun (www.geoffreylitt.com)
    • diff --git a/docs/log.txt b/docs/log.txt index 3623e868cbd..58e7aec8e4b 100644 --- a/docs/log.txt +++ b/docs/log.txt @@ -1,41 +1,45 @@ -2024/12/26 17:13:51 error parsing https://mikehudack.substack.com/feed: http error: 403 Forbidden -2024/12/26 17:13:51 Fetched posts from https://mikehudack.substack.com/feed, took 38.05705ms -2024/12/26 17:13:51 error parsing https://themargins.substack.com/feed.xml: http error: 403 Forbidden -2024/12/26 17:13:51 Fetched posts from https://themargins.substack.com/feed.xml, took 39.502926ms -2024/12/26 17:13:51 Fetched posts from https://www.slowernews.com/rss.xml, took 56.610997ms -2024/12/26 17:13:51 Fetched posts from https://macwright.com/rss.xml, took 81.738996ms -2024/12/26 17:13:51 error parsing https://highgrowthengineering.substack.com/feed: http error: 403 Forbidden -2024/12/26 17:13:51 Fetched posts from https://highgrowthengineering.substack.com/feed, took 81.996547ms -2024/12/26 17:13:51 Fetched posts from https://www.benkuhn.net/index.xml, took 83.663094ms -2024/12/26 17:13:51 Fetched posts from https://anewsletter.alisoneroman.com/feed, took 92.700474ms -2024/12/26 17:13:51 Fetched posts from https://twobithistory.org/feed.xml, took 144.356807ms -2024/12/26 17:13:51 Fetched posts from https://jvns.ca/atom.xml, took 247.272105ms -2024/12/26 17:13:51 Fetched posts from https://scattered-thoughts.net/rss.xml, took 257.328517ms -2024/12/26 17:13:51 Fetched posts from https://danluu.com/atom.xml, took 337.021171ms -2024/12/26 17:13:51 Fetched posts from https://routley.io/reserialised/great-expectations/2022-08-24/index.xml, took 342.705425ms -2024/12/26 17:13:51 Fetched posts from https://www.wildlondon.org.uk/blog/all/rss.xml, took 416.683616ms -2024/12/26 17:13:51 Fetched posts from https://blog.golang.org/feed.atom?format=xml, took 484.320994ms -2024/12/26 17:13:52 Content still empty after HTML reader: https://pbat.ch/recurse/demos/trio/ -2024/12/26 17:13:52 Fetched posts from https://blog.veitheller.de/feed.rss, took 1.093347344s -2024/12/26 17:13:52 Fetched posts from https://commoncog.com/blog/rss/, took 1.422476668s -2024/12/26 17:13:53 Fetched posts from http://tonsky.me/blog/atom.xml, took 1.783959557s -2024/12/26 17:13:53 Content still empty after HTML reader: https://todaythings.substack.com/p/to-acquire-a-goshawk-is-a-major-decision -2024/12/26 17:13:54 Content still empty after HTML reader: https://yc-map.vercel.app/ -2024/12/26 17:13:54 Fetched posts from https://gochugarugirl.com/feed/, took 3.191369954s -2024/12/26 17:13:55 Content still empty after HTML reader: http://tinylogger.com/max/wnTJ9xu3fw5UiXLp -2024/12/26 17:13:56 Content still empty after HTML reader: https://www.youtube.com/watch?v=IQqtsm-bBRU -2024/12/26 17:13:56 Content still empty after HTML reader: https://papermatch.mitanshu.tech/ -2024/12/26 17:13:57 Fetched posts from https://solar.lowtechmagazine.com/feeds/all-en.atom.xml, took 6.161083787s -2024/12/26 17:13:57 Get "https://www.fixbrowser.org/": tls: failed to verify certificate: x509: certificate signed by unknown authority -2024/12/26 17:13:58 Fetched posts from https://blaggregator.recurse.com/atom.xml?token=4c4c4e40044244aab4a36e681dfb8fb0, took 6.796726752s -2024/12/26 17:13:58 Fetched posts from https://hnrss.org/frontpage?points=50, took 6.861782873s -2024/12/26 17:14:01 Fetched posts from https://joy.recurse.com/feed.atom, took 10.10695938s -2024/12/26 17:14:21 error parsing https://rachelbythebay.com/w/atom.xml: Get "https://rachelbythebay.com/w/atom.xml": dial tcp 216.218.228.215:443: i/o timeout -2024/12/26 17:14:21 Fetched posts from https://rachelbythebay.com/w/atom.xml, took 30.002896543s -2024/12/26 17:14:21 Skipping writing post, no content: https://pbat.ch/recurse/demos/trio/ -2024/12/26 17:14:21 Skipping writing post, no content: https://yc-map.vercel.app/ -2024/12/26 17:14:21 Skipping writing post, no content: https://www.youtube.com/watch?v=IQqtsm-bBRU -2024/12/26 17:14:21 Skipping writing post, no content: https://papermatch.mitanshu.tech/ -2024/12/26 17:14:21 Skipping writing post, no content: https://todaythings.substack.com/p/to-acquire-a-goshawk-is-a-major-decision -2024/12/26 17:14:21 Skipping writing post, no content: http://tinylogger.com/max/wnTJ9xu3fw5UiXLp -2024/12/26 17:14:21 Templated 41 posts, took 5.687389ms +2024/12/26 18:22:08 error parsing https://www.slowernews.com/rss.xml: Get "https://www.slowernews.com/rss.xml": read tcp 10.1.0.15:47448->34.234.106.80:443: read: connection reset by peer +2024/12/26 18:22:08 Fetched posts from https://www.slowernews.com/rss.xml, took 29.570348ms +2024/12/26 18:22:08 error parsing https://highgrowthengineering.substack.com/feed: http error: 403 Forbidden +2024/12/26 18:22:08 Fetched posts from https://highgrowthengineering.substack.com/feed, took 39.395463ms +2024/12/26 18:22:08 error parsing https://themargins.substack.com/feed.xml: http error: 403 Forbidden +2024/12/26 18:22:08 Fetched posts from https://themargins.substack.com/feed.xml, took 41.331424ms +2024/12/26 18:22:08 error parsing https://mikehudack.substack.com/feed: http error: 403 Forbidden +2024/12/26 18:22:08 Fetched posts from https://mikehudack.substack.com/feed, took 45.20789ms +2024/12/26 18:22:08 Fetched posts from https://macwright.com/rss.xml, took 52.970227ms +2024/12/26 18:22:09 Fetched posts from https://www.benkuhn.net/index.xml, took 74.067428ms +2024/12/26 18:22:09 Fetched posts from https://jvns.ca/atom.xml, took 146.360995ms +2024/12/26 18:22:09 Fetched posts from https://joy.recurse.com/feed.atom, took 163.368665ms +2024/12/26 18:22:09 Fetched posts from https://anewsletter.alisoneroman.com/feed, took 166.168548ms +2024/12/26 18:22:09 Fetched posts from https://twobithistory.org/feed.xml, took 176.730036ms +2024/12/26 18:22:09 Fetched posts from https://www.wildlondon.org.uk/blog/all/rss.xml, took 257.420833ms +2024/12/26 18:22:09 Fetched posts from https://scattered-thoughts.net/rss.xml, took 282.34156ms +2024/12/26 18:22:09 Fetched posts from https://routley.io/reserialised/great-expectations/2022-08-24/index.xml, took 316.160215ms +2024/12/26 18:22:09 Fetched posts from https://danluu.com/atom.xml, took 334.148238ms +2024/12/26 18:22:09 Content still empty after HTML reader: https://pbat.ch/recurse/demos/trio/ +2024/12/26 18:22:09 Fetched posts from https://blog.golang.org/feed.atom?format=xml, took 474.852675ms +2024/12/26 18:22:09 Fetched posts from https://blog.veitheller.de/feed.rss, took 571.127446ms +2024/12/26 18:22:09 Fetched posts from https://solar.lowtechmagazine.com/feeds/all-en.atom.xml, took 902.580145ms +2024/12/26 18:22:09 Fetched posts from http://tonsky.me/blog/atom.xml, took 917.87782ms +2024/12/26 18:22:10 Content still empty after HTML reader: https://vrklovespaper.substack.com/p/software-for-stationery-lovers +2024/12/26 18:22:10 Content still empty after HTML reader: https://todaythings.substack.com/p/to-acquire-a-goshawk-is-a-major-decision +2024/12/26 18:22:10 Fetched posts from https://commoncog.com/blog/rss/, took 1.92245796s +2024/12/26 18:22:11 Content still empty after HTML reader: https://yc-map.vercel.app/ +2024/12/26 18:22:11 Fetched posts from https://gochugarugirl.com/feed/, took 2.580740115s +2024/12/26 18:22:11 Content still empty after HTML reader: http://tinylogger.com/max/wnTJ9xu3fw5UiXLp +2024/12/26 18:22:12 Content still empty after HTML reader: https://www.youtube.com/watch?v=IQqtsm-bBRU +2024/12/26 18:22:12 Content still empty after HTML reader: https://papermatch.mitanshu.tech/ +2024/12/26 18:22:13 Content still empty after HTML reader: https://supersnowflakemaker.com +2024/12/26 18:22:13 Fetched posts from https://hnrss.org/frontpage?points=50, took 4.716998076s +2024/12/26 18:22:13 Fetched posts from https://blaggregator.recurse.com/atom.xml?token=4c4c4e40044244aab4a36e681dfb8fb0, took 4.953835159s +2024/12/26 18:22:38 error parsing https://rachelbythebay.com/w/atom.xml: Get "https://rachelbythebay.com/w/atom.xml": dial tcp 216.218.228.215:443: i/o timeout +2024/12/26 18:22:38 Fetched posts from https://rachelbythebay.com/w/atom.xml, took 30.004337455s +2024/12/26 18:22:38 Skipping writing post, no content: https://vrklovespaper.substack.com/p/software-for-stationery-lovers +2024/12/26 18:22:38 Skipping writing post, no content: https://pbat.ch/recurse/demos/trio/ +2024/12/26 18:22:38 Skipping writing post, no content: https://yc-map.vercel.app/ +2024/12/26 18:22:38 Skipping writing post, no content: https://www.youtube.com/watch?v=IQqtsm-bBRU +2024/12/26 18:22:38 Skipping writing post, no content: https://papermatch.mitanshu.tech/ +2024/12/26 18:22:38 Skipping writing post, no content: https://todaythings.substack.com/p/to-acquire-a-goshawk-is-a-major-decision +2024/12/26 18:22:38 Skipping writing post, no content: https://supersnowflakemaker.com +2024/12/26 18:22:38 Skipping writing post, no content: http://tinylogger.com/max/wnTJ9xu3fw5UiXLp +2024/12/26 18:22:38 Templated 43 posts, took 5.811262ms diff --git a/docs/posts/a-minecraft-server-written-in-cobol.html b/docs/posts/a-minecraft-server-written-in-cobol.html index 8e1bd8017cc..b94599be6b1 100644 --- a/docs/posts/a-minecraft-server-written-in-cobol.html +++ b/docs/posts/a-minecraft-server-written-in-cobol.html @@ -17,11 +17,11 @@ Original

    A Minecraft server written in COBOL

    -
    +

    Build Test

    A Minecraft server written in COBOL. It supports Minecraft 1.21.4 (the latest version at time of writing).

    -

    Features

    +

    The following features are already working:

    • infinite terrain generation and dynamic chunk loading
      • @@ -50,7 +50,7 @@

      A Minecraft server written in COBOL

    • trapdoors (including interaction)
    • beds
    -

    How-to

    +

    CobolCraft was developed using GnuCOBOL and is meant to be run on Linux. Support for other operating systems such as Windows has not been tested. However, it is possible to use Docker for a platform-independent deployment.

    @@ -101,7 +101,7 @@

    A Minecraft server written in COBOL

    To make it accessible from the outside (your local network, via VPN, port forwarding, on a rented server, ...), you can start the Docker container like this:

    docker run --rm -it -p 0.0.0.0:25565:25565 meyfa/cobolcraft
    -

    Why?

    +

    Well, there are quite a lot of rumors and stigma surrounding COBOL. This intrigued me to find out more about this language, which is best done with some sort of project, in my opinion. You heard right - I had no prior COBOL experience going into this.

    @@ -115,18 +115,18 @@

    A Minecraft server written in COBOL

    https://gnucobol.sourceforge.io/HTML/gnucobpg.html

    To learn more about the Minecraft protocol, you can refer to https://wiki.vg/Protocol. In some cases, it may be helpful to look at real server traffic to better understand the flow of information.

    -

    Program Overview

    +

    This section provides a high-level overview of CobolCraft from a software design viewpoint.

    -

    Source Components

    +

    The program entrypoint is main.cob. The remaining COBOL sources are located in the src/ directory, including src/server.cob, which contains the bulk of CobolCraft.

    These sources are located in the cpp/ directory and get compiled into a shared library (.so on Linux).

    TCP sockets are managed by the CBL_GC_SOCKET socket library located in the CBL_GC_SOCKET/ directory.

    -

    Packet Blobs

    +

    CobolCraft makes use of network data captured from an instance of the official server application via Wireshark. This data is located in the blobs/ directory and is decoded at run-time.

    -

    Data Extraction

    +

    The official Minecraft (Java Edition) server and client applications contain large amounts of data such as:

    • block and item types
      • @@ -137,7 +137,7 @@

      A Minecraft server written in COBOL

      The CobolCraft Makefile has a target that downloads the .jar and extracts the JSON data from it. The JSON files are evaluated at runtime using a custom-built generic JSON parser, such that CobolCraft can inter-operate successfully with the Minecraft client without distributing potentially copyrighted material.

      -

      Legal Notices

      +

      This project (except 3rd-party contents as stated below) is licensed under the MIT License. See LICENSE for further information.

      This project includes the 3rd-party CBL_GC_SOCKET shared library, licensed under the LGPL v3. diff --git a/docs/posts/a-new-learning-experience-on-mdn.html b/docs/posts/a-new-learning-experience-on-mdn.html new file mode 100644 index 00000000000..166edc59511 --- /dev/null +++ b/docs/posts/a-new-learning-experience-on-mdn.html @@ -0,0 +1,53 @@ + + + + + + + James Routley | Feed + + + + Back + Original +

      A new learning experience on MDN

      + +
      A new learning experience on MDN title. Common programming syntax used in CSS, HTML, Bash, and a star. A browser window in the top right with some code and a lightbulb signifying learning. +

      A major update to the MDN Learn Web Development section started in November 2024 and was finally published in December 2024. +To summarize, the MDN Curriculum has been merged into Learn Web Development. +This post looks at the background leading up to this decision being made, what the changes mean specifically, and what updates you can expect to see in the future.

      Overview of learning material on MDN

      We originally launched the MDN Learn Web Development section in 2016 with the aim of making MDN more accessible to non-experts and helping to take new web developers from "beginner to comfortable".

      +

      The content was pretty successful — by 2019 it was being used by over a million people per month to learn web development topics. +However, it was noted that the structure was sub-par:

      +
        +
      1. The content had become bloated with topics that weren't really suitable for beginner web developers — either they were too advanced, or they were out of scope altogether.
        2. +
      2. Beginners tend to want a robust pathway they can follow to get the knowledge they need, rather than being expected to figure out what to learn and when.
        3. +
      3. Learners these days tend to want interactive multimedia content, not just text.
        4. +

      Developing the MDN curriculum

      To solve the second issue highlighted above, we created a resource to help guide people towards learning a better skillset, making them more employable, and enabling them to build a better, more accessible, more responsible web of tomorrow.

      +

      As part of this project, we did substantial research to find out exactly what skills are seen as essential in new hires, and what the most common knowledge gaps are. +The resulting curriculum was intended to be useful as a study guide for self-learners, and a syllabus for educators to base courses on. +We also used it as a place to experiment with including interactive multimedia content via our learning partner, Scrimba. +We launched the MDN Curriculum in early 2024.

      Why the update?

      The curriculum was well-received by educators, but we quickly received feedback that users found it confusing having two learning resources on MDN, with the curriculum/learning pathway in one place and the learning content in another place.

      +

      In addition, the pathway was just a curriculum; learners still had to figure out what content to look at to achieve the learning objectives. And it did nothing to solve the first issue listed above — the content was still bloated.

      The new state of learning material on MDN

      +

      To provide a less confusing, more streamlined learning experience, we decided to merge the curriculum pathway into the MDN Learn Web Development section, restructuring it in the process. The results can be seen at the following new URL — developer.mozilla.org/docs/Learn_web_development.

      + +

      The most significant changes are as follows:

      +
        +
      • The articles now follow the same structure as the curriculum, with useful background and environment setup information in Getting started, the web fundamentals everyone should know in Core, and optional extra topics in Extensions. There is a clear pathway to follow between each article in the first two major sections, so readers know what to learn next at each stage.
        • +
      • In some cases, content was deemed not suitable for a beginner audience and has been repurposed as extensions/additional articles or migrated to other parts of MDN.
        • +
      • The specific learning outcomes detailed in the curriculum have been added to the top of the Getting started, Core, and some of the Extension articles, to match the learning outcomes detailed in the Curriculum.
        • +
      • Other features from the Curriculum have been migrated across to the Learn Web Development section, such as the About page and Resources for educators.
        • +
      +

      Initially, the Curriculum section will stay, however it will be merged into the Learn area over the next few iterations of this work and will be removed when it is felt the time is right. +We will keep a downloadable version as a resource for educators.

      What's next for learners on MDN

      Moving forward, we will continue to update the content and design to make Learn Web Development even more useful to learners and educators.

      +

      We are intending to give the article content a significant overhaul as we move into 2025. A lot of the content is timeless and does a good job of teaching the fundamentals, but the pacing is uneven, some of the articles are pretty long (which can be intimidating for beginners), and some of the challenges and examples have been around for a long time. Inspired by resources such as the content produced by our learning partner, Scrimba, we intend to shake our content up a bit to make it more fun, bite-size, and digestible.

      +

      We are also looking at improving the design of our learning pages, to echo the bright bold design we used on the curriculum and improve the experience further. +You can expect to see regular iterative improvements going forward, so watch this space.

      Summary

      We hope you find the new Learn Web Development content structure useful — have a look around and let us know what you think.

      +

      If you still have unanswered questions or wish to report issues, please get in touch via the usual communication channels. +If your issue concerns a specific piece of content, you might want to file a GitHub issue.

      Previous Post Countdown to the holidays with daily coding challenges

      + + diff --git a/docs/posts/bad-research-idea-false-statements-in-e-graphs.html b/docs/posts/bad-research-idea-false-statements-in-e-graphs.html index e2107bf66ab..ee62fcbefe2 100644 --- a/docs/posts/bad-research-idea-false-statements-in-e-graphs.html +++ b/docs/posts/bad-research-idea-false-statements-in-e-graphs.html @@ -23,7 +23,7 @@

      bad research idea: false statements in e-graphs

      OK after much squinting at the progression of rewrite rules... I think I have found an example of where the logic goes wrong.

      Can you spot the error?

      -Screenshot 2024-12-23 at 10 06 52 PM +Screenshot 2024-12-23 at 10 06 52 PM

      The issue here is that the empty int list TupleInt.EMPTY is unified with TupleInt(0, partial(lambda i, self, j: Int.if_(j == self.length(), i, self[j])), 101, TupleInt.empty) aka TupleInt(0, lambda j: Int.if_(j == 0, 101, TupleInt.EMPTY[j])))

      Now let's say we do a naive index the empty list like TupleInt.EMPTY[0]. We could say this incorrect, or how we can represent it is that it unifies with Int.NEVER. But it can show up in the e-graph, because in if_ conditionals, the false branch can end up doing indexing that is not allowed. So we want it to not mess things up.

      And in this case then, it will evaluate to (lambda j: Int.if_(j == 0, 101, TupleInt.EMPTY[j])))(0) which is Int.if_(0 == 0, 101, TupleInt.EMPTY[0])) which is 101... So then what we get is that 101 is unified with Int.NEVER which... isn't good! Is really bad! Because it means all numbers can be unified together basically, i.e. false is true whatever.

      diff --git a/docs/posts/cognitive-load-is-what-matters.html b/docs/posts/cognitive-load-is-what-matters.html new file mode 100644 index 00000000000..e75e3631488 --- /dev/null +++ b/docs/posts/cognitive-load-is-what-matters.html @@ -0,0 +1,305 @@ + + + + + + + James Routley | Feed + + + + Back + Original +

      Cognitive load is what matters

      + +
      +

      +

      The logo image was taken from Reddit.

      +

      It is a living document, last update: November 2024. Your contributions are welcome!

      +

      + Introduction +

      +

      There are so many buzzwords and best practices out there, but let's focus on something more fundamental. What matters is the amount of confusion developers feel when going through the code.

      +

      Confusion costs time and money. Confusion is caused by high cognitive load. It's not some fancy abstract concept, but rather a fundamental human constraint.

      +

      Since we spend far more time reading and understanding code than writing it, we should constantly ask ourselves whether we are embedding excessive cognitive load into our code.

      +

      + Cognitive load +

      +
      +

      Cognitive load is how much a developer needs to think in order to complete a task.

      +
      +

      When reading code, you put things like values of variables, control flow logic and call sequences into your head. The average person can hold roughly four such chunks in working memory. Once the cognitive load reaches this threshold, it becomes much harder to understand things.

      +

      Let's say we have been asked to make some fixes to a completely unfamiliar project. We were told that a really smart developer had contributed to it. Lots of cool architectures, fancy libraries and trendy technologies were used. In other words, the author had created a high cognitive load for us.

      +

      Cognitive Load

      +

      We should reduce the cognitive load in our projects as much as possible.

      +

      + Types of cognitive load +

      +

      Intrinsic - caused by the inherent difficulty of a task. It can't be reduced, it's at the very heart of software development.

      +

      Extraneous - created by the way the information is presented. Caused by factors not directly relevant to the task, such as smart author's quirks. Can be greatly reduced. We will focus on this type of cognitive load.

      +

      Intrinsic vs Extraneous

      + +

      Let's jump straight to the concrete practical examples of extraneous cognitive load.

      +

      We will refer to the level cognitive load as follows:

      +
      +

      Our brain is much more complex and unexplored, but we can go with this simplistic model.

      +
      +

      + Complex conditionals +

      +
      if val > someConstant 
+    && (condition2 || condition3) 
+    && (condition4 && !condition5) { 
+    ...
+}
+
      +

      Introduce intermediate variables with meaningful names:

      +
      isValid = val > someConstant
+isAllowed = condition2 || condition3
+isSecure = condition4 && !condition5
+
+if isValid && isAllowed && isSecure {
+    ...
+}
+
      + +

      + Nested ifs +

      +
      if isValid { 
+    if isSecure { 
+        stuff // 🧠+++
+    }
+}
+
      +

      Compare it with the early returns:

      +
      if !isValid
+    return
+
+if !isSecure
+    return
+
+
+
+stuff 
+
      +

      We can focus on the happy path only, thus freeing our working memory from all sorts of preconditions.

      +

      + Inheritance nightmare +

      +

      We are asked to change a few things for our admin users: 🧠

      + +
      AdminController extends UserController extends GuestController extends BaseController
      + +

      Ohh, part of the functionality is in BaseController, let's have a look: 🧠+

      +

      Oh, wait, there's SuperuserController which extends AdminController. By modifying AdminController we can break things in the inherited class, so let's dive in SuperuserController first: 🤯

      +

      Prefer composition over inheritance. We won't go into detail - there's plenty of material out there.

      +

      + Too many small methods, classes or modules +

      +
      +

      Method, class and module are interchangeable in this context

      +
      +

      Mantras like "methods should be shorter than 15 lines of code" or "classes should be small" turned out to be somewhat wrong.

      +

      Deep module - simple interface, complex functionality

      +

      Deep module

      +

      Having too many shallow modules can make it difficult to understand the project. Not only do we have to keep in mind each module responsibilities, but also all their interactions. To understand the purpose of a shallow module, we first need to look at the functionality of all the related modules. 🤯

      +
      +

      Information hiding is paramount, and we don't hide as much complexity in shallow modules.

      +
      +

      I have two pet projects, both of them are somewhat 5K lines of code. The first one has 80 shallow classes, whereas the second one has only 7 deep classes. I haven't been maintaining any of these projects for one year and a half.

      +

      Once I came back, I realised that it was extremely difficult to untangle all the interactions between those 80 classes in the first project. I would have to rebuild an enormous amount of cognitive load before I could start coding. On the other hand, I was able to grasp the second project quickly, because it had only a few deep classes with a simple interface.

      +
      +

      The best components are those that provide powerful functionality yet have simple interface.

      +
      +

      The interface of the UNIX I/O is very simple. It has only five basic calls:

      +
      open(path, flags, permissions)
+read(fd, buffer, count)
+write(fd, buffer, count)
+lseek(fd, offset, referencePosition)
+close(fd)
+
      +

      A modern implementation of this interface has hundreds of thousands of lines of code. Lots of complexity is hidden under the hood. Yet it is easy to use due to its simple interface.

      +
      +

      This deep module example is taken from the book A Philosophy of Software Design by John K. Ousterhout. Not only does this book cover the very essence of complexity in software development, but it also has the greatest interpretation of Parnas' influential paper On the Criteria To Be Used in Decomposing Systems into Modules. Both are essential reads. Other related readings: It's probably time to stop recommending Clean Code, Small Functions considered Harmful.

      +
      +

      P.S. If you think we are rooting for bloated God objects with too many responsibilities, you got it wrong.

      +

      + Shallow modules and SRP +

      +

      All too often, we end up creating lots of shallow modules, following some vague "a module should be responsible for one, and only one, thing" principle. What is this blurry one thing? Instantiating an object is one thing, right? So MetricsProviderFactoryFactory seems to be just fine. The names and interfaces of such classes tend to be more mentally taxing than their entire implementations, what kind of abstraction is that? Something went wrong.

      +
      +

      Jumping between such shallow components is mentally exhausting, linear thinking is more natural to us humans.

      +
      +

      We make changes to our systems to satisfy our users and stakeholders. We are responsible to them.

      +
      +

      A module should be responsible to one, and only one, user or stakeholder.

      +
      +

      This is what this Single Responsibility Principle is all about. Simply put, if we introduce a bug in one place, and then two different business people come to complain, we've violated the principle. It has nothing to do with the number of things we do in our module.

      +

      But even now, this interpretation can do more harm than good. This rule can be understood in as many different ways as there are individuals. A better approach would be to look at how much cognitive load it all creates. It's mentally demanding to remember that change in one module can trigger a chain of reactions across different business streams. And that's about it.

      +

      + Too many shallow microservices +

      +

      This shallow-deep module principle is scale-agnostic, and we can apply it to microservices architecture. Too many shallow microservices won't do any good - the industry is heading towards somewhat "macroservices", i.e., services that are not so shallow (=deep). One of the worst and hardest to fix phenomena is so-called distributed monolith, which is often the result of this overly granular shallow separation.

      +

      I once consulted a startup where a team of five developers introduced 17(!) microservices. They were 10 months behind schedule and appeared nowhere close to the public release. Every new requirement led to changes in 4+ microservices. Diagnostic difficulty in integration space skyrocketed. Both time to market and cognitive load were unacceptably high. 🤯

      + +

      Is this the right way to approach the uncertainty of a new system? It's enormously difficult to elicit the right logical boundaries in the beginning. The key is to make decisions as late as you can responsibly wait, because that is when you have the most information on which to base the decision. By introducing a network layer up front, we make our design decisions hard to revert right from the start. The team's only justification was: "The FAANG companies proved microservices architecture to be effective". Hello, you got to stop dreaming big.

      +

      The Tanenbaum-Torvalds debate argued that Linux's monolithic design was flawed and obsolete, and that a microkernel architecture should be used instead. Indeed, the microkernel design seemed to be superior "from a theoretical and aesthetical" point of view. On the practical side of things - three decades on, microkernel-based GNU Hurd is still in development, and monolithic Linux is everywhere. This page is powered by Linux, your smart teapot is powered by Linux. By monolithic Linux.

      +

      A well-crafted monolith with truly isolated modules is often much more flexible than a bunch of microservices. It also requires far less cognitive effort to maintain. It's only when the need for separate deployments becomes crucial, such as scaling the development team, that you should consider adding a network layer between the modules, future microservices.

      +

      + Feature-rich languages +

      +

      We feel excited when new features got released in our favourite language. We spend some time learning these features, we build code upon them.

      +

      If there are lots of features, we may spend half an hour playing with a few lines of code, to use one or another feature. And it's kind of a waste of time. But what's worse, when you come back later, you would have to recreate that thought process!

      +

      You not only have to understand this complicated program, you have to understand why a programmer decided this was the way to approach a problem from the features that are available. 🤯

      +

      These statements are made by none other than Rob Pike.

      +
      +

      Reduce cognitive load by limiting the number of choices.

      +
      +

      Language features are OK, as long as they are orthogonal to each other.

      +
      + Thoughts from an engineer with 20 years of C++ experience ⭐️ +

      I was looking at my RSS reader the other day and noticed that I have somewhat three hundred unread articles under the "C++" tag. I haven't read a single article about the language since last summer, and I feel great!

      +

      I've been using C++ for 20 years for now, that's almost two-thirds of my life. Most of my experience lies in dealing with the darkest corners of the language (such as undefined behaviours of all sorts). It's not a reusable experience, and it's kind of creepy to throw it all away now.

      +

      Like, can you imagine, the token || has a different meaning in requires ((!P<T> || !Q<T>)) and in requires (!(P<T> || Q<T>)). The first is the constraint disjunction, the second is the good-old logical or operator, and they behave differently.

      +

      You can't allocate space for a trivial type and just memcpy a set of bytes there without extra effort - that won't start the lifetime of an object. This was the case before C++20. It was fixed in C++20, but the cognitive load of the language has only increased.

      +

      Cognitive load is constantly growing, even though things got fixed. I should know what was fixed, when it was fixed, and what it was like before. I am a professional after all. Sure, C++ is good at legacy support, which also means that you will face that legacy. For example, last month a colleague of mine asked me about some behaviour in C++03. 🤯

      +

      There were 20 ways of initialization. Uniform initialization syntax has been added. Now we have 21 ways of initialization. By the way, does anyone remember the rules for selecting constructors from the initializer list? Something about implicit conversion with the least loss of information, but if the value is known statically, then... 🤯

      +

      This increased cognitive load is not caused by a business task at hand. It is not an intrinsic complexity of the domain. It is just there due to historical reasons (extraneous cognitive load).

      +

      I had to come up with some rules. Like, if that line of code is not as obvious and I have to remember the standard, I better not write it that way. The standard is somewhat 1500 pages long, by the way.

      +

      By no means I am trying to blame C++. I love the language. It's just that I am tired now.

      +
      +

      + Business logic and HTTP status codes +

      +On the backend we return: +
        +
      • 401 for expired jwt token
        • +
      • 403 for not enough access
        • +
      • 418 for banned users
        • +
      +

      The guys on the frontend use backend API to implement login functionality. They would have to temporarily create the following cognitive load in their brains:

      +
        +
      • 401 is for expired jwt token // 🧠+, ok just temporary remember it
        • +
      • 403 is for not enough access // 🧠++
        • +
      • 418 is for banned users // 🧠+++
        • +
      +

      Frontend developers would (hopefully) introduce some kind numeric status -> meaning dictionary on their side, so that subsequent generations of contributors wouldn't have to recreate this mapping in their brains.

      +

      Then QA people come into play: + "Hey, I got 403 status, is that expired token or not enough access?" + QA people can't jump straight to testing, because first they have to recreate the cognitive load that the guys on the backend once created.

      +

      Why hold this custom mapping in our working memory? It's better to abstract away your business details from the HTTP transfer protocol, and return self-descriptive codes directly in the response body:

      +
      {
+    "code": "jwt_has_expired"
+}
+
      +

      Cognitive load on the frontend side: 🧠 (fresh, no facts are held in mind)

      +

      The same rule applies to all sorts of numeric statuses (in the database or wherever) - prefer self-describing strings. We are not in the era of 640K computers to optimise for memory.

      +
      +

      People spend time arguing between 401 and 403, making decisions based on their own mental models. New developers are coming in, and they need to recreate that thought process. You may have documented the "whys" (ADRs) for your code, helping newcomers to understand the decisions made. But in the end it just doesn't make any sense. We can separate errors into either user-related or server-related, but apart from that, things are kind of blurry.

      +
      +

      P.S. It's often mentally taxing to distinguish between "authentication" and "authorization". We can use simpler terms like "login" and "permissions" to reduce the cognitive load.

      +

      + Abusing DRY principle +

      +

      Do not repeat yourself - that is one of the first principles you are taught as a software engineer. It is so deeply embedded in ourselves that we can not stand the fact of a few extra lines of code. Although in general a good and fundamental rule, when overused it leads to the cognitive load we can not handle.

      +

      Nowadays, everyone builds software based on logically separated components. Often those are distributed among multiple codebases representing separate services. When you strive to eliminate any repetition, you might end up creating tight coupling between unrelated components. As a result changes in one part may have unintended consequences in other seemingly unrelated areas. It can also hinder the ability to replace or modify individual components without impacting the entire system. 🤯

      +

      In fact, the same problem arises even within a single module. You might extract common functionality too early, based on perceived similarities that might not actually exist in the long run. This can result in unnecessary abstractions that are difficult to modify or extend.

      +

      Rob Pike once said:

      +
      +

      A little copying is better than a little dependency.

      +
      +

      We are tempted to not reinvent the wheel so strong that we are ready to import large, heavy libraries to use a small function that we could easily write by ourselves.

      +

      All your dependencies are your code. Going through 10+ levels of stack trace of some imported library and figuring out what went wrong (because things go wrong) is painful.

      + +

      + Tight coupling with a framework +

      +

      There's a lot of "magic" in frameworks. By relying too heavily on a framework, we force all upcoming developers to learn that "magic" first. It can take months. Even though frameworks enable us to launch MVPs in a matter of days, in the long run they tend to add unnecessary complexity and cognitive load.

      +

      Worse yet, at some point frameworks can become a significant constraint when faced with a new requirement that just doesn't fit the architecture. From here onwards people end up forking a framework and maintaining their own custom version. Imagine the amount of cognitive load a newcomer would have to build (i.e. learn this custom framework) in order to deliver any value. 🤯

      +

      By no means do we advocate to invent everything from scratch!

      +

      We can write code in a somewhat framework-agnostic way. The business logic should not reside within a framework; rather, it should use the framework's components. Put a framework outside of your core logic. Use the framework in a library-like fashion. This would allow new contributors to add value from day one, without the need of going through debris of framework-related complexity first.

      +
      + Why I Hate Frameworks +
      +

      + Layered architecture +

      +

      There is a certain engineering excitement about all this stuff.

      +

      I myself was a passionate advocate of Hexagonal/Onion Architecture for years. I used it here and there and encouraged other teams to do so. The complexity of our projects went up, the sheer number of files alone had doubled. It felt like we were writing a lot of glue code. On ever changing requirements we had to make changes across multiple layers of abstractions, it all became tedious. 🤯

      +

      Abstraction is supposed to hide complexity, here it just adds indirection. Jumping from call to call to read along and figure out what goes wrong and what is missing is a vital requirement to quickly solve a problem. With this architecture’s layer uncoupling it requires an exponential factor of extra, often disjointed, traces to get to the point where the failure occurs. Every such trace takes space in our limited working memory. 🤯

      +

      This architecture was something that made intuitive sense at first, but every time we tried applying it to projects it made a lot more harm than good. In the end, we gave it all up in favour of the good old dependency inversion principle. No port/adapter terms to learn, no unnecessary layers of horizontal abstractions, no extraneous cognitive load.

      +

      If you think that such layering will allow you to quickly replace a database or other dependencies, you're mistaken. Changing the storage causes lots of problems, and believe us, having some abstractions for the data access layer is the least of your worries. At best, abstractions can save somewhat 10% of your migration time (if any), the real pain is in data model incompatibilities, communication protocols, distributed systems challenges, and implicit interfaces.

      +
      +

      +With a sufficient number of users of an API,

      +
      +

      We did a storage migration, and that took us about 10 months. The old system was single-threaded, so the exposed events were sequential. All our systems depended on that observed behaviour. This behavior was not part of the API contract, it was not reflected in the code. A new distributed storage didn't have that guarantee - the events came out-of-order. We spent only a few hours coding a new storage adapter. We spent the next 10 months on dealing with out-of-order events and other challenges. It's now funny to say that layering helps us replace components quickly.

      +

      So, why pay the price of high cognitive load for such a layered architecture, if it doesn't pay off in the future? Plus, in most cases, that future of replacing some core component never happens.

      +

      These architectures are not fundamental, they are just subjective, biased consequences of more fundamental principles. Why rely on those subjective interpretations? Follow the fundamental rules instead: dependency inversion principle, cognitive load and information hiding. Discuss.

      +

      Do not add layers of abstractions for the sake of an architecture. Add them whenever you need an extension point that is justified for practical reasons. Layers of abstraction aren't free of charge, they are to be held in our working memory.

      +

      + DDD +

      +

      Domain-driven design has some great points, although it is often misinterpreted. People say "We write code in DDD", which is a bit strange, because DDD is about problem space, not about solution space.

      +

      Ubiquitous language, domain, bounded context, aggregate, event storming are all about problem space. They are meant to help us learn the insights about the domain and extract the boundaries. DDD enables developers, domain experts and business people to communicate effectively using a single, unified language. Rather than focusing on these problem space aspects of DDD, we tend to emphasise particular folder structures, services, repositories, and other solution space techniques.

      +

      Chances are that the way we interpret DDD is likely to be unique and subjective. And if we build code upon this understanding, i.e., if we create a lot of extraneous cognitive load - future developers are doomed. 🤯

      + +

      + Examples +

      + + + +

      + These architectures are quite boring and easy to understand. Anyone can grasp them without much mental effort. +

      + +

      + Involve junior developers in architecture reviews. They will help you to identify the mentally demanding areas. +

      + +

      + Cognitive load in familiar projects +

      +
      +

      The problem is that familiarity is not the same as simplicity. They feel the same — that same ease of moving through a space without much mental effort — but for very different reasons. Every “clever” (read: “self-indulgent”) and non-idiomatic trick you use incurs a learning penalty for everyone else. Once they have done that learning, then they will find working with the code less difficult. So it is hard to recognise how to simplify code that you are already familiar with. This is why I try to get “the new kid” to critique the code before they get too institutionalised!

      +

      It is likely that the previous author(s) created this huge mess one tiny increment at a time, not all at once. So you are the first person who has ever had to try to make sense of it all at once.

      +

      In my class I describe a sprawling SQL stored procedure we were looking at one day, with hundreds of lines of conditionals in a huge WHERE clause. Someone asked how anyone could have let it get this bad. I told them: “When there are only 2 or 3 conditionals, adding another one doesn’t make any difference. By the time there are 20 or 30 conditionals, adding another one doesn’t make any difference!”

      +

      There is no “simplifying force” acting on the code base other than deliberate choices that you make. Simplifying takes effort, and people are too often in a hurry.

      +

      Thanks to Dan North for his comment.

      +
      +

      If you've internalized the mental models of the project into your long-term memory, you won't experience a high cognitive load.

      +

      Mental Models

      +

      The more mental models there are to learn, the longer it takes for a new developer to deliver value.

      +

      Once you onboard new people on your project, try to measure the amount of confusion they have (pair programming may help). If they're confused for more than ~40 minutes in a row - you've got things to improve in your code.

      +

      If you keep the cognitive load low, people can contribute to your codebase within the first few hours of joining your company.

      + +

      + Conclusion +

      +

      Imagine for a moment that what we inferred in the second chapter isn’t actually true. If that’s the case, then the conclusion we just negated, along with the conclusions in the previous chapter that we had accepted as valid, might not be correct either. 🤯

      +

      Do you feel it? Not only do you have to jump all over the article to get the meaning (shallow modules!), but the paragraph in general is difficult to understand. We have just created an unnecessary cognitive load in your head. Do not do this to your colleagues.

      + +

      Smart Author

      +

      We should reduce any cognitive load above and beyond what is intrinsic to the work we do.

      + +
      + + + + + +
      + + diff --git a/docs/posts/crt-simulation-in-a-gpu-shader-looks-better-than-black-frame-insertion.html b/docs/posts/crt-simulation-in-a-gpu-shader-looks-better-than-black-frame-insertion.html deleted file mode 100644 index c8ee3a5bd90..00000000000 --- a/docs/posts/crt-simulation-in-a-gpu-shader-looks-better-than-black-frame-insertion.html +++ /dev/null @@ -1,63 +0,0 @@ - - - - - - - James Routley | Feed - - - - Back - Original -

      CRT Simulation in a GPU Shader, Looks Better Than Black Frame Insertion

      - -
      -

      Please enable cookies.

      -
      -

      - -

      You are unable to access blurbusters.com

      -

      - - - -
      -
      -
      -

      Why have I been blocked?

      - -

      This website is using a security service to protect itself from online attacks. The action you just performed triggered the security solution. There are several actions that could trigger this block including submitting a certain word or phrase, a SQL command or malformed data.

      -
      - -
      -

      What can I do to resolve this?

      - -

      You can email the site owner to let them know you were blocked. Please include what you were doing when this page came up and the Cloudflare Ray ID found at the bottom of this page.

      -
      -
      -
      - -

      - Cloudflare Ray ID: 8f82b4577f1acf25 - • - - Your IP: - - 52.234.47.212 - • - - Performance & security by Cloudflare - -

      - - -
      -
      - - diff --git a/docs/posts/f-a-proof-oriented-general-purpose-programming-language.html b/docs/posts/f-a-proof-oriented-general-purpose-programming-language.html index 1a4b451011e..871a0acf900 100644 --- a/docs/posts/f-a-proof-oriented-general-purpose-programming-language.html +++ b/docs/posts/f-a-proof-oriented-general-purpose-programming-language.html @@ -48,13 +48,13 @@

      F*: A proof oriented general purpose programming language

    An online - book Proof-oriented + book Proof-oriented Programming In F* is being written and regular updates are posted online. You probably want to read it while trying out examples and exercises in your browser by clicking the image below.

    - F* Tutorial + F* Tutorial

    Low*

    We also have a @@ -68,21 +68,21 @@

    Course Material

  • Embedding Proof-oriented Programming Languages in F*)
  • Formal Verification with F* and Meta-F*
  • Verifying Low-Level Code for Correctness and Security
    • @@ -135,7 +135,7 @@

    Project Everest

    F* is an active topic of research, both in the programming languages and formal methods community, as well as from an application perspective in the security and systems communities. - We list a few of them below, with full citations to these papers available in this bibliography. + We list a few of them below, with full citations to these papers available in this bibliography. If you would like your paper included in this list, please contact fstar-maintainers@googlegroups.com. diff --git a/docs/posts/how-to-build-an-electrically-heated-table.html b/docs/posts/how-to-build-an-electrically-heated-table.html index bc128f52409..edee60192c1 100644 --- a/docs/posts/how-to-build-an-electrically-heated-table.html +++ b/docs/posts/how-to-build-an-electrically-heated-table.html @@ -18,40 +18,32 @@

    How to Build an Electrically Heated Table?

    +
    +
    -
    -
    - - -
    -
    -
    - Image: The electrically heated table that we build in this manual. Photo: Marina Kálcheva. Model: Anita Filippova. -
    -
    - Image: The electrically heated table that we build in this manual. Photo: Marina Kálcheva. Model: Anita Filippova. - - - -

    +

    +
    +
    +Image: The electrically heated table that we build in this manual. Photo: Marina Kálcheva. Model: Anita Filippova. +
    +
    +Image: The electrically heated table that we build in this manual. Photo: Marina Kálcheva. Model: Anita Filippova. + + + +

    View original image - + View dithered image -

    -
    -
    -
    +

    +
    +
    +
    - - - - - -
    • Why build an electrically heated table?
    • The table @@ -82,24 +74,24 @@

      The heated table is an excellent example of our ancestors’ energy-efficient way of warming: heating people, not spaces. Historically, glowing charcoal from the fireplace heated the space under the table. While that provided sufficient warmth, it also carried a significant risk of fire and carbon monoxide poisoning. Nowadays, we can use electric heating technology instead. For example, the Japanese kotatsu is still in use, but it’s now working with a small electric heater fixed under the table surface.

      In this manual, I will walk you through the making of an electrically heated work desk for one person. I have built the table for myself in the co-working space in Barcelona where I have my office now. The building, an old industrial warehouse, has very high ceilings, no insulation, and little sun in winter. It can get very cold here and conventional heating systems don’t work. My heated table turns out to be a perfect solution. I can power it with a solar panel, a wind turbine, a bike generator, or a battery. I can also plug it into the power grid.

      -
      - Image: The author sitting at the heated table in his workspace, where the experiments took place. Over the table is a large wool blanket with an equally large cotton blanket on top of it. Photo: Marina Kálcheva. -
      -
      - Image: The author sitting at the heated table in his workspace, where the experiments took place. Over the table is a large wool blanket with an equally large cotton blanket on top of it. Photo: Marina Kálcheva. - - - -

      +

      +Image: The author sitting at the heated table in his workspace, where the experiments took place. Over the table is a large wool blanket with an equally large cotton blanket on top of it. Photo: Marina Kálcheva. +
      +
      +Image: The author sitting at the heated table in his workspace, where the experiments took place. Over the table is a large wool blanket with an equally large cotton blanket on top of it. Photo: Marina Kálcheva. + + + +

      View original image - + View dithered image -

      -
      -
      -
      +

      +
      +
      +

      A heated table offers exceptional comfort. The lower part of your body gets immersed in heat as if you are baking in the sun or sitting in a hot bath. The warmth quickly spreads to the rest of your body through the bloodstream.

      During a week of experiments in December 2024, with indoor air temperatures of 12-14°C (53-57°F), I recorded very low energy use for my freshly built heated table: between 50 and 75 watt-hours per hour. 1 Compare that to a conventional electric portable heater, which easily consumes 1,500 watt-hours per hour (and does not guarantee thermal comfort). My heated table uses as little electricity as charging a laptop or heating two liters of water for a hot water bottle (58 watt-hours per hour, assuming you reheat the water every two hours).

      @@ -116,86 +108,86 @@

      The manual

    • Extra insulation material (I used cork)
    -
    - Image: How to assemble an electrically heated and insulated table. 1. Fix carbon heat foil to thin wood board 2. Fix wood board to table 3. Add cork insulation between wood board and table 4. Add blanket. Illustration: Marie Verdeil. -
    -
    - Image: How to assemble an electrically heated and insulated table. 1. Fix carbon heat foil to thin wood board 2. Fix wood board to table 3. Add cork insulation between wood board and table 4. Add blanket. Illustration: Marie Verdeil. - - - -

    +

    +Image: How to assemble an electrically heated and insulated table. 1. Fix carbon heat foil to thin wood board 2. Fix wood board to table 3. Add cork insulation between wood board and table 4. Add blanket. Illustration: Marie Verdeil. +
    +
    +Image: How to assemble an electrically heated and insulated table. 1. Fix carbon heat foil to thin wood board 2. Fix wood board to table 3. Add cork insulation between wood board and table 4. Add blanket. Illustration: Marie Verdeil. + + + +

    View original image - + View dithered image -

    -
    -
    -
    +

    +
    +
    +
    -
    - Image: How to assemble an electrically heated and insulated table. Illustration: Marie Verdeil. -
    -
    - Image: How to assemble an electrically heated and insulated table. Illustration: Marie Verdeil. - - - -

    +

    +Image: How to assemble an electrically heated and insulated table. Illustration: Marie Verdeil. +
    +
    +Image: How to assemble an electrically heated and insulated table. Illustration: Marie Verdeil. + + + +

    View original image - + View dithered image -

    -
    -
    -
    +

    +
    +
    +
    -
    - How to assemble an electrically heated and insulated table. Illustration: Marie Verdeil. -
    -
    - How to assemble an electrically heated and insulated table. Illustration: Marie Verdeil. - - - -

    +

    +How to assemble an electrically heated and insulated table. Illustration: Marie Verdeil. +
    +
    +How to assemble an electrically heated and insulated table. Illustration: Marie Verdeil. + + + +

    View original image - + View dithered image -

    -
    -
    -
    +

    +
    +
    +

    Step 1: Get a table

    This manual concerns a table for only one person - my writing desk. Unlike the Japanese and Middle Eastern examples, my table is adapted to a Western-style sitting position: not on the floor but on a chair. You can turn any table into a personal heat source, but some are better suited than others. Most importantly, you should be able to screw a flat heating foil (step 2) under the table. However, structural elements may complicate that, as is the case for my table (see the image below).

    -
    - Image: The table before the conversion. Photo: Kris De Decker. -
    -
    - Image: The table before the conversion. Photo: Kris De Decker. - - - -

    +

    +Image: The table before the conversion. Photo: Kris De Decker. +
    +
    +Image: The table before the conversion. Photo: Kris De Decker. + + + +

    View original image - + View dithered image -

    -
    -
    -
    +

    +
    +
    +

    I solved this by installing the heating foil on a thin wooden board which I then screwed against the supporting elements. However, for some other tables, this may not work. Choose a wooden table. Wood insulates relatively well, so a wooden table top already provides some of the insulation you need to maximize heat production. It’s easy to screw things onto a wooden table as well.

    You can build a larger heated table that can seat more people, but in that case, you will have to connect several heating foils (step 2) and stitch several blankets (step 8) together. Low-tech Magazine will build a large heated table for several people during a workshop in Barcelona on January 25, 2025.

    @@ -204,24 +196,24 @@

    Ste

    Carbon heating film

    The best heating element for an electrically heated table - and the one I am using in this manual - is carbon or infrared heating film. These very thin heating foils are primarily meant for electric floor and wall heating in buildings and vehicles, for protecting batteries or water tanks against the cold, or for warming beehives and terrariums. Infrared heating films are low-temperature, large-surface heaters, so there’s no risk of burns or fire through direct contact with skin or clothes. They are meant to operate at a maximum temperature of 40-45°C (104-113°F).

    -
    - Image: The infrared heating foil, screwed against a thin wood board, ready to be fixed below the table surface. Photo: Kris De Decker. -
    -
    - Image: The infrared heating foil, screwed against a thin wood board, ready to be fixed below the table surface. Photo: Kris De Decker. - - - -

    +

    +Image: The infrared heating foil, screwed against a thin wood board, ready to be fixed below the table surface. Photo: Kris De Decker. +
    +
    +Image: The infrared heating foil, screwed against a thin wood board, ready to be fixed below the table surface. Photo: Kris De Decker. + + + +

    View original image - + View dithered image -

    -
    -
    -
    +

    +
    +
    +

    Voltage

    Carbon heating foils come in different voltages: 12V, 24V, and 110/220V. I chose a 12V heating foil to make my table compatible with my 12V solar installations and bike generator. If you have a 24V renewable power system, opt for a 24V heating foil.

    @@ -236,24 +228,24 @@

    Step 3:

    Step 4: Wire everything together

    The thermostat is connected between the heating foil and the power source, as shown in the illustration below. The wiring may be different for other thermostat models.

    -
    - Image: How to wire the thermostat to the heating foil and the power source. 1. Thermostat 2. Fuse 3. Temperature sensor 4. Carbon heat film. Illustration by Marie Verdeil. -
    -
    - Image: How to wire the thermostat to the heating foil and the power source. 1. Thermostat 2. Fuse 3. Temperature sensor 4. Carbon heat film. Illustration by Marie Verdeil. - - - -

    +

    +Image: How to wire the thermostat to the heating foil and the power source. 1. Thermostat 2. Fuse 3. Temperature sensor 4. Carbon heat film. Illustration by Marie Verdeil. +
    +
    +Image: How to wire the thermostat to the heating foil and the power source. 1. Thermostat 2. Fuse 3. Temperature sensor 4. Carbon heat film. Illustration by Marie Verdeil. + + + +

    View original image - + View dithered image -

    -
    -
    -
    +

    +
    +
    +

    Cable size

    Your cables must be thick enough for the current that flows through them. Infrared heating foils are sold with thick electric cables included, and they are often much longer than you need them to be. You can cut them shorter and use the rest to wire the whole system. If you want to use other cables, then use a multimeter to measure the current that the heating film draws. For example, my 12V heat foil requires 6.6 Amps, so my cables - in the complete circuit - should have a conductor cross-section of at least 2.63 mm2 (that’s 13 AWG gauge, check this chart).

    @@ -266,24 +258,24 @@

    Temperature sensor

    Where does the thermostat go?

    Before you start wiring the system, decide where your thermostat goes, because it will determine the length of the cables you need. I have my thermostat installed under the table, hidden under the blankets. Once programmed, there’s no need to access it regularly (see further). Having the thermostat on top of the table means that you need a hole in your blanket for all the cables to go through. It also complicates the adding and removing of blankets.

    -
    - Image: The thermostat at the side of the table, wired to the heat foil and the power source. Photo: Marina Kálcheva. -
    -
    - Image: The thermostat at the side of the table, wired to the heat foil and the power source. Photo: Marina Kálcheva. - - - -

    +

    +Image: The thermostat at the side of the table, wired to the heat foil and the power source. Photo: Marina Kálcheva. +
    +
    +Image: The thermostat at the side of the table, wired to the heat foil and the power source. Photo: Marina Kálcheva. + + + +

    View original image - + View dithered image -

    -
    -
    -
    +

    +
    +
    +

    Step 5: Program the thermostat

    To program the thermostat, connect it to the power source. First, select the “heating” function. My thermostat’s default setting was “cooling” and I struggled to make it work at first. Here’s the steps to follow:

    @@ -316,24 +308,24 @@

    Step 8: Find

    The energy efficiency and thermal comfort of an electrically heated table are in large part determined by the type and size of the blankets you put over it. The blankets form part of the heat film insulation layer, but if they are long enough to reach the ground they also trap warm air under the table.

    Radiant heating systems transfer energy to surfaces - including your body - and do not warm up the air directly. However, the air temperature under the table will slowly increase indirectly due to the higher surface temperatures of the blankets, the table, the carpet, and the person who sits at the table. During the experiments, the air temperature below my table at 25 cm above the floor increased by about 10°C (18°F).

    -
    - Image: The electrically heated table featuring a 240x240cm Abbruzzo wool blanket. Photo: Marina Kálcheva. -
    -
    - Image: The electrically heated table featuring a 240x240cm Abbruzzo wool blanket. Photo: Marina Kálcheva. - - - -

    +

    +Image: The electrically heated table featuring a 240x240cm Abbruzzo wool blanket. Photo: Marina Kálcheva. +
    +
    +Image: The electrically heated table featuring a 240x240cm Abbruzzo wool blanket. Photo: Marina Kálcheva. + + + +

    View original image - + View dithered image -

    -
    -
    -
    +

    +
    +
    +

    Wool

    Choose a wool blanket. Wool traps heat very efficienctly, is much more flame-resistant than other textile materials, doesn’t get dirty or smelly easily, regulates humidity, and purifies the air. 4 New wool blankets can be pricey at hundreds of euros for the size we need. However, I purchased four second-hand wool blankets for 90 euros, two of them large enough to reach the ground. If you find a wool blanket that is ugly or stained, simply layer it with a nicer and cheaper cotton blanket.

    @@ -342,24 +334,24 @@

    Blanket size

    Work surface

    Although it feels nice to work on a wool or cotton surface in winter, you can also put a wooden board on top of the blanket, cut to size, in order to protect the blanket from wear and dirt. Or, you drape a cotton tablecloth over the blanket, which is easier to wash than wool.

    -
    - Image: The electrically heated table, ready to get dressed with various blankets. Photo: Marina Kálcheva. -
    -
    - Image: The electrically heated table, ready to get dressed with various blankets. Photo: Marina Kálcheva. - - - -

    +

    +Image: The electrically heated table, ready to get dressed with various blankets. Photo: Marina Kálcheva. +
    +
    +Image: The electrically heated table, ready to get dressed with various blankets. Photo: Marina Kálcheva. + + + +

    View original image - + View dithered image -

    -
    -
    -
    +

    +
    +
    +

    Dressing and undressing the table

    Any heating system must be able to be adjusted to achieve the desired comfort. For a central heating system, that happens by manipulating the thermostat. However, that doesn’t work so well for a heated table, because the temperature range of the carbon heating film is limited. Going below 38°C (100°F) will not provide a pleasant sensation of warmth, while prolonged heating above 45°C (113°F) may damage the heating film and make it too hot to touch. However, you can adjust thermal comfort in a wide range of air temperatures by “dressing” and “undressing” the table: by adding and removing textile layers, and by using them in different ways.

    @@ -368,44 +360,44 @@

    - Image: Anita Filippova works as an intern at the heated table in Low-tech Magazine&rsquo;s office. There is an extra blanket wrapped around her shoulders and the chair. Photo: Marina Kálcheva. -
    -
    - Image: Anita Filippova works as an intern at the heated table in Low-tech Magazine’s office. There is an extra blanket wrapped around her shoulders and the chair. Photo: Marina Kálcheva. - - - -

    +

    +Image: Anita Filippova works as an intern at the heated table in Low-tech Magazine&rsquo;s office. There is an extra blanket wrapped around her shoulders and the chair. Photo: Marina Kálcheva. +
    +
    +Image: Anita Filippova works as an intern at the heated table in Low-tech Magazine’s office. There is an extra blanket wrapped around her shoulders and the chair. Photo: Marina Kálcheva. + + + +

    View original image - + View dithered image -

    -
    -
    -
    +

    +
    +
    +

    -
    - Image: Anita Filippova works as an intern at the heated table in Low-tech Magazine&rsquo;s office. It&rsquo;s not an exaggeration to say that the blanket becomes part of your clothing. Photo: Marina Kálcheva. -
    -
    - Image: Anita Filippova works as an intern at the heated table in Low-tech Magazine’s office. It’s not an exaggeration to say that the blanket becomes part of your clothing. Photo: Marina Kálcheva. - - - -

    +

    +Image: Anita Filippova works as an intern at the heated table in Low-tech Magazine&rsquo;s office. It&rsquo;s not an exaggeration to say that the blanket becomes part of your clothing. Photo: Marina Kálcheva. +
    +
    +Image: Anita Filippova works as an intern at the heated table in Low-tech Magazine’s office. It’s not an exaggeration to say that the blanket becomes part of your clothing. Photo: Marina Kálcheva. + + + +

    View original image - + View dithered image -

    -
    -
    -
    +

    +
    +
    +

    However, as you add more blankets, the textile layer becomes increasingly heavy, and it takes more time to get in and out - to “dress” and “undress”. Consequently, a lighter cover is preferable if it provides sufficient comfort. On a chilly spring evening, one or two shorter blankets may be sufficient to keep you comfortable. I have tested the table like that and it did almost as good as with a large blanket. Energy use was somewhat higher and thermal comfort somewhat lower - I especially noted cold feet. However, should it be a bit warmer, I would prefer this setup because it’s more practical to get up from the desk.

    Overheating can be solved without reaching for the thermostat as well: lift a corner of the carpet with your foot and let some heat escape, or remove one of the blankets. It would be more energy-efficient to turn down the thermostat, but the energy use of a heated table is already so low that there is room for some convenience. The comfort of a heated table can be further improved by a heated chair or bench, or by putting a screen behind your chair, covered with heat foil on one side and with insulation on the other side. That’s something for a future manual.

    @@ -437,57 +429,38 @@

    Relevant books:

  • Heating people, not spaces.
    • -
    - Image: Powering the heated table with the bike generator. Photo: Marina Kálcheva. -
    -
    - Image: Powering the heated table with the bike generator. Photo: Marina Kálcheva. - - - -

    +

    +Image: Powering the heated table with the bike generator. Photo: Marina Kálcheva. +
    +
    +Image: Powering the heated table with the bike generator. Photo: Marina Kálcheva. + + + +

    View original image - + View dithered image -

    -
    -
    -
    +

    +
    +
    +
    +
    - - -
    - - - - - - - + - - - - +
    -
    - +
    + - - - - -
    + +
    diff --git a/docs/posts/into-cps-never-to-return.html b/docs/posts/into-cps-never-to-return.html index bf0ce915bd6..c235f52601d 100644 --- a/docs/posts/into-cps-never-to-return.html +++ b/docs/posts/into-cps-never-to-return.html @@ -71,7 +71,7 @@

    How do I…?

    Our cps function will take two arguments. The first argument, exp, is the expression to compile. The second argument, k, is a continuation. We have to do something with our values, but CPS requires that functions never -returns. So what do we do? Call another function, of course.

    +return. So what do we do? Call another function, of course.

    This means that the top-level invocation of cps will be passed some useful top-level continuation like print-to-screen or write-to-file. All child @@ -88,7 +88,7 @@

    How do I…?

    While you totally can generate real first-class functions for use as continuations, it can often be useful to partition your CPS IR by separating them. All real (user) functions will take a continuation as a last -parameter—for handing off their return values— and can arbitrarily escape, +parameter—for handing off their return values—and can arbitrarily escape, whereas all continuations are generated and allocated/freed in a stack-like manner. (We could even implement them using a native stack if we wanted. See “Partitioned CPS” and “Recovering the stack” from Might’s page.)

    diff --git a/docs/posts/portspoof-emulate-a-valid-service-on-all-65535-tcp-ports.html b/docs/posts/portspoof-emulate-a-valid-service-on-all-65535-tcp-ports.html index 9350cd9e8cc..89f543c41ab 100644 --- a/docs/posts/portspoof-emulate-a-valid-service-on-all-65535-tcp-ports.html +++ b/docs/posts/portspoof-emulate-a-valid-service-on-all-65535-tcp-ports.html @@ -17,244 +17,44 @@ Original

    Portspoof: Emulate a valid service on all 65535 TCP ports

    -
    -

    Every open TCP port emulates a service

    -

    Portspoof has a huge database of dynamic service signatures, that will be used to generate fake banners and fool scanners.

    -

    Scanning software usually tries to determine a service version that is running on an open port. Portspoof will respond to every service probe with a valid service signature, that is dynamically generated based on a service signature regular expression database.

    -

    As a result an attacker will not be able to determine which port numbers your system is truly using:

    -
     **`nmap -F -sV 127.0.0.1`**
- Starting Nmap 6.47 ( http://nmap.org )
- Stats: 0:00:30 elapsed; 0 hosts completed (1 up), 1 undergoing Service Scan
- Nmap scan report for 127.0.0.1
- Host is up (0.21s latency).
- PORT      STATE SERVICE          VERSION
- 7/tcp     open  http             Milestone XProtect video surveillance http interface (tu-ka)
- 9/tcp     open  ntop-http        Ntop web interface 1ey (Q)
- 13/tcp    open  ftp              VxWorks ftpd 6.a
- 21/tcp    open  http             Grandstream VoIP phone http config 6193206
- 22/tcp    open  http             Cherokee httpd X
- 23/tcp    open  ftp              MacOS X Server ftpd (MacOS X Server 790751705)
- 25/tcp    open  smtp?
- 26/tcp    open  http             ZNC IRC bouncer http config 0.097 or later
- 37/tcp    open  finger           NetBSD fingerd
- 53/tcp    open  ftp              Rumpus ftpd
- 79/tcp    open  http             Web e (Netscreen administrative web server)
- 80/tcp    open  http             BitTornado tracker dgpX
- 81/tcp    open  hosts2-ns?
- 88/tcp    open  http             3Com OfficeConnect Firewall http config
- 106/tcp   open  pop3pw?
- 110/tcp   open  ipp              Virata-EmWeb nbF (HP Laserjet 4200 TN http config)
- 111/tcp   open  imap             Dovecot imapd
- 113/tcp   open  smtp             Xserve smtpd
- 119/tcp   open  nntp?
- 135/tcp   open  http             netTALK Duo http config
- 139/tcp   open  http             Oversee Turing httpd kC (domain parking)
- 143/tcp   open  crestron-control TiVo DVR Crestron control server
- 144/tcp   open  http             Ares Galaxy P2P httpd 7942927
- 179/tcp   open  http             WMI ViH (3Com 5500G-EI switch http config)
- 199/tcp   open  smux?
- 389/tcp   open  http-proxy       ziproxy http proxy
- 427/tcp   open  vnc              (protocol 3)
- 443/tcp   open  https?
- 444/tcp   open  snpp?
- 445/tcp   open  http             Pogoplug HBHTTP QpwKdZQ
- 465/tcp   open  http             Gordian httpd 322410 (IQinVision IQeye3 webcam rtspd)
- 513/tcp   open  login?
- 514/tcp   open  finger           ffingerd
- 515/tcp   open  pop3             Eudora Internet Mail Server X pop3d 4918451
- 543/tcp   open  ftp              Dell Laser Printer z printer ftpd k
- 544/tcp   open  ftp              Solaris ftpd
- 548/tcp   open  http             Medusa httpd Elhmq (Sophos Anti-Virus Home http config)
- 554/tcp   open  rtsp?
- 587/tcp   open  http-proxy       Pound http proxy
- 631/tcp   open  efi-webtools     EFI Fiery WebTools communication
- 646/tcp   open  ldp?
- 873/tcp   open  rsync?
- 990/tcp   open  http             OpenWrt uHTTPd
- 993/tcp   open  ftp              Konica Minolta bizhub printer ftpd
- 995/tcp   open  pop3s?
- 1025/tcp  open  sip-proxy        Comdasys SIP Server D
- 1026/tcp  open  LSA-or-nterm?
- 1027/tcp  open  IIS?
- 1028/tcp  open  rfidquery        Mercury3 RFID Query protocol
- 1029/tcp  open  smtp-proxy       ESET NOD32 anti-virus smtp proxy
- 1110/tcp  open  http             qhttpd
- 1433/tcp  open  http             ControlByWeb WebRelay-Quad http admin
- 1720/tcp  open  H.323/Q.931?
- 1723/tcp  open  pptp?
- 1755/tcp  open  http             Siemens Simatic HMI MiniWeb httpd
- 1900/tcp  open  tunnelvision     Tunnel Vision VPN info 69853
- 2000/tcp  open  telnet           Patton SmartNode 4638 VoIP adapter telnetd
- 2001/tcp  open  dc?
- 2049/tcp  open  nfs?
- 2121/tcp  open  http             Bosch Divar Security Systems http config
- 2717/tcp  open  rtsp             Darwin Streaming Server 104621400
- 3000/tcp  open  pop3             Solid pop3d
- 3128/tcp  open  irc-proxy        muh irc proxy
- 3306/tcp  open  ident            KVIrc fake identd
- 3389/tcp  open  ms-wbt-server?
- 3986/tcp  open  mapper-ws_ethd?
- 4899/tcp  open  printer          QMC DeskLaser printer (Status o)
- 5000/tcp  open  http             D-Link DSL-eTjM http config
- 5009/tcp  open  airport-admin?
- 5051/tcp  open  ssh              (protocol 325257)
- 5060/tcp  open  http             apt-cache/apt-proxy httpd
- 5101/tcp  open  ftp              OKI BVdqeC-ykAA VoIP adapter ftpd kHttKI
- 5190/tcp  open  http             Conexant-EmWeb JqlM (Intertex IX68 WAP http config; SIPGT TyXT)
- 5357/tcp  open  wsdapi?
- 5432/tcp  open  postgresql?
- 5631/tcp  open  irc              ircu ircd
- 5666/tcp  open  litecoin-jsonrpc Litecoin JSON-RPC f_
- 5800/tcp  open  smtp             Lotus Domino smtpd rT Beta y
- 5900/tcp  open  ftp
- 6000/tcp  open  http             httpd.js (Songbird WebRemote)
- 6001/tcp  open  daap             mt-daapd DAAP TGeiZA
- 6646/tcp  open  unknown
- 7070/tcp  open  athinfod         Athena athinfod
- 8000/tcp  open  amanda           Amanda backup system index server (broken: libsunmath.so.1 not found)
- 8008/tcp  open  http?
- 8009/tcp  open  ajp13?
- 8080/tcp  open  http             D-Link DGL-4300 WAP http config
- 8081/tcp  open  http             fec ysp (Funkwerk bintec R232B router; .h.K...z)
- 8443/tcp  open  smtp
- 8888/tcp  open  smtp             OpenVMS smtpd uwcDNI (OpenVMS RVqcGIr; Alpha)
- 9100/tcp  open  jetdirect?
- 9999/tcp  open  http             Embedded HTTPD 3BOzejtHW (Netgear MRd WAP http config; j)
- 10000/tcp open  http             MikroTik router http config (RouterOS 0982808)
- 32768/tcp open  filenet-tms?
- 49152/tcp open  unknown
- 49153/tcp open  http             ASSP Anti-Spam Proxy httpd XLgR(?)?
- 49154/tcp open  http             Samsung AllShare httpd
- 49155/tcp open  ftp              Synology DiskStation NAS ftpd
- 49156/tcp open  aspi             ASPI server 837305
- 49157/tcp open  sip              AVM FRITZ!Box |
-
    -

    By using those two techniques together:

    -
      -
    • your attackers will have a tough time while trying to identify your real services.
      • -
    • the only way to determine if a service is emulated is through a protocol probe (imagine probing protocols for 65k open ports!).
      • -
    • it takes more than 8hours and 200MB of sent data in order to properly go through the reconessaince phase for your system ( nmap -sV -p - equivalent).
      • +

      Ugh, totally failed at getting these #happy posts out. But here is one more post before the year is over. Some of these links are from recent, and some are from drafts I created but never published.

      +

      Listening

      + -

      *** Art of Active (Offensive) Defense***

      -

      Portspoof can be used as an 'Exploitation Framework Frontend', that turns your system into responsive and aggressive machine. In practice this usually means exploiting your attackers' tools and exploits... -At the moment there are few example exploits in the configuration file (portspoof.conf)

      -

      Portspoof is meant to be a lightweight, fast, portable and secure addition to any firewall system or security system.

      -

      The general goal of the program is to make the reconessaince phase slow and bothersome for your attackers as much it is only possible. -This is quite a change to the standard 5s Nmap scan, that will give a full view of your systems running services.

      -

      The most important features that this software has:

      -
        -
      • it will add some real pain to your attackers reconessaince phase.
        • -
      • it is a userland software and does not require root privileges !
        • -
      • it binds to just ONE tcp port per a running instance !
        • -
      • it is easily customizable through your iptables rules
        • -
      • marginal CPU and memory usage (multithreaded)
        • -
      • more than 9000 dynamic service signatures to feed your attackers scanning software !
        • +

        Watching

        + -
      +

      Reading

      + +

      Playing

      +
        +
      • onedr0p/exportarr is a cool piece of software which scrapes a variety of Servarr type services and surfaces their data in a prometheus endpoint.
        • +
      • I've been hooked on Dragon Age: The Veilguard. It's got a lot of people complaining about it, but I'm loving the story telling. I particularly love how much of the story is hidden in side quests. I feel like I have to really search to understand what is happening, and am always rewarded for it.
        • +
      +

      /Nat

    diff --git a/docs/posts/server-sent-events-sse-are-underrated.html b/docs/posts/server-sent-events-sse-are-underrated.html deleted file mode 100644 index c2df86ab484..00000000000 --- a/docs/posts/server-sent-events-sse-are-underrated.html +++ /dev/null @@ -1,312 +0,0 @@ - - - - - - - James Routley | Feed - - - - Back - Original -

    Server-Sent Events (SSE) Are Underrated

    - -
    - - - - - - - - - -

    - - - -

    - - -

    Most developers know about WebSockets, but Server-Sent Events (SSE) offer a simpler, often overlooked alternative that deserves more attention. Let's explore why this technology is underrated and how it can benefit your applications.

    -

    What are Server-Sent Events?

    SSE establishes a one-way communication channel from server to client over HTTP. Unlike WebSockets' bidirectional connection, SSE maintains an open HTTP connection for server-to-client updates. Think of it as a radio broadcast: the server (station) transmits, and clients (receivers) listen.

    -

    Why are they Underrated?

    Two main factors contribute to SSE's underappreciation:

    -
      -
    1. WebSocket's Popularity: WebSockets' full-duplex communication capabilities often overshadow SSE's simpler approach
      2. -
    2. Perceived Limitations: The unidirectional nature might seem restrictive, though it's often sufficient for many use cases
      3. -
    -

    Key Strengths of SSE

    1. Implementation Simplicity

    SSE leverages standard HTTP protocols, eliminating the complexity of WebSocket connection management.

    -

    2. Infrastructure Compatibility

    SSE works seamlessly with existing HTTP infrastructure:

    -
      -
    • Load balancers
      • -
    • Proxies
      • -
    • Firewalls
      • -
    • Standard HTTP servers
      • -
    -

    As pointed out in the HN comment section, some issues may occur:

    -
    -

    I'd be careful with that assumption. I have tried using SSE through some 3rd party load balancer at my work and it doesn't work that well. Because SSE is long-lived and doesn't normally close immediately, this load balancer will keep collecting and collecting bytes from the server and not forward it until server closes the connection, effectively making SSEs useless. I had to use WebSockets instead to get around this limitation with the load balancer.

    -
    -

    3. Resource Efficiency

    Lower resource consumption compared to WebSockets due to:

    -
      -
    • Unidirectional nature
      • -
    • Standard HTTP connection usage
      • -
    • No persistent socket maintenance
      • -
    -

    4. Automatic Reconnection

    Built-in browser support for:

    -
      -
    • Connection interruption handling
      • -
    • Automatic reconnection attempts
      • -
    • Resilient real-time experience
      • -
    -

    5. Clear Semantics

    One-way communication pattern enforces:

    -
      -
    • Clear separation of concerns
      • -
    • Straightforward data flow
      • -
    • Simplified application logic
      • -
    -

    Practical Applications

    SSE excels in these scenarios:

    -
      -
    1. Real-time News Feeds and Social Updates
      2. -
    2. Stock Tickers and Financial Data
      3. -
    3. Progress Bars and Task Monitoring
      4. -
    4. Server Logs Streaming
      5. -
    5. Collaborative Editing (for updates)
      6. -
    6. Gaming Leaderboards
      7. -
    7. Location Tracking Systems
      8. -
    -

    Implementation Examples

    Server-Side (Flask)

    from flask import Flask, Response, stream_with_context
-import time
-import random
-
-app = Flask(__name__)
-
-def generate_random_data():
-    while True:
-        data = f"data: Random value: {random.randint(1, 100)}\n\n"
-        yield data
-        time.sleep(1)
-
-@app.route('/stream')
-def stream():
-    return Response(
-        stream_with_context(generate_random_data()),
-        mimetype='text/event-stream'
-    )
-
-if __name__ == '__main__':
-    app.run(debug=True)
-
    -

    Client-Side (JavaScript)

    const eventSource = new EventSource("/stream");
-
-eventSource.onmessage = function(event) {
-    const dataDiv = document.getElementById("data");
-    dataDiv.innerHTML += `<p>${event.data}</p>`;
-};
-
-eventSource.onerror = function(error) {
-    console.error("SSE error:", error);
-};
-
    -

    Code Explanation

    Server-Side Components:

      -
    • /stream route handles SSE connections
      • -
    • generate_random_data() continuously yields formatted events
      • -
    • text/event-stream mimetype signals SSE protocol
      • -
    • stream_with_context maintains Flask application context
      • -
    -

    Client-Side Components:

      -
    • EventSource object manages SSE connection
      • -
    • onmessage handler processes incoming events
      • -
    • onerror handles connection issues
      • -
    • Automatic reconnection handled by browser
      • -
    -
    -

    Like the article so far? Subscribe to the blog so you don’t miss the next part -

    - -
    -

    Limitations and Considerations

    When implementing SSE, be aware of these constraints:

    -

    1. Unidirectional Communication

      -
    • Server-to-client only
      • -
    • Requires separate HTTP requests for client-to-server communication
      • -
    -

    2. Browser Support

      -
    • Well-supported in modern browsers
      • -
    • May need polyfills for older browsers
      • -
    -

    3. Data Format

      -
    • Primary support for text-based data
      • -
    • Binary data requires encoding (e.g., Base64)
      • -
    -

    Best Practices

      -
    1. Error Handling
      2. -
    -
    eventSource.onerror = function(error) {
-    if (eventSource.readyState === EventSource.CLOSED) {
-        console.log("Connection was closed");
-    }
-};
-
    -
      -
    1. Connection Management
      2. -
    -
    // Clean up when done
-function closeConnection() {
-    eventSource.close();
-}
-
    -
      -
    1. Reconnection Strategy
      2. -
    -
    let retryAttempts = 0;
-const maxRetries = 5;
-
-eventSource.onclose = function() {
-    if (retryAttempts < maxRetries) {
-        setTimeout(() => {
-            // Reconnect logic
-            retryAttempts++;
-        }, 1000 * retryAttempts);
-    }
-};
-
    -

    Real-World Example: ChatGPT's Implementation

    Modern Language Learning Models (LLMs) utilize Server-Sent Events (SSE) for streaming responses. Let's explore how these implementations work and what makes them unique.

    -

    The General Pattern

    All major LLM providers implement streaming using a common pattern:

    -
      -
    • Return content-type: text/event-stream header
      • -
    • Stream data blocks separated by \r\n\r\n
      • -
    • Each block contains a data: JSON line
      • -
    -

    Important Note

    While SSE typically works with the browser's EventSource API, LLM implementations can't use this directly because:

    -
      -
    • EventSource only supports GET requests
      • -
    • LLM APIs require POST requests
      • -
    -

    OpenAI Implementation

    Basic Request Structure

    curl https://api.openai.com/v1/chat/completions \
-  -H "Content-Type: application/json" \
-  -H "Authorization: Bearer $OPENAI_API_KEY" \
-  -d '{
-    "model": "gpt-4o-mini",
-    "messages": [{"role": "user", "content": "Hello, world?"}],
-    "stream": true,
-    "stream_options": {
-      "include_usage": true
-    }
-  }'
-
    -

    Response Format

    Each chunk follows this structure:

    -
    "data":{
-   "id":"chatcmpl-AiT7GQk8zzYSC0Q8UT1pzyRzwxBCN",
-   "object":"chat.completion.chunk",
-   "created":1735161718,
-   "model":"gpt-4o-mini-2024-07-18",
-   "system_fingerprint":"fp_0aa8d3e20b",
-   "choices":[
-      {
-         "index":0,
-         "delta":{
-            "content":"!"
-         },
-         "logprobs":null,
-         "finish_reason":null
-      }
-   ],
-   "usage":null
-}
-
-"data":{
-   "id":"chatcmpl-AiT7GQk8zzYSC0Q8UT1pzyRzwxBCN",
-   "object":"chat.completion.chunk",
-   "created":1735161718,
-   "model":"gpt-4o-mini-2024-07-18",
-   "system_fingerprint":"fp_0aa8d3e20b",
-   "choices":[
-      {
-         "index":0,
-         "delta":{
-            
-         },
-         "logprobs":null,
-         "finish_reason":"stop"
-      }
-   ],
-   "usage":null
-}
-
    -

    Key headers returned by OpenAI:

    -
    HTTP/2 200
-date: Wed, 25 Dec 2024 21:21:59 GMT
-content-type: text/event-stream; charset=utf-8
-access-control-expose-headers: X-Request-ID
-openai-organization: user-esvzealexvl5nbzmxrismbwf
-openai-processing-ms: 100
-openai-version: 2020-10-01
-x-ratelimit-limit-requests: 10000
-x-ratelimit-limit-tokens: 200000
-x-ratelimit-remaining-requests: 9999
-x-ratelimit-remaining-tokens: 199978
-x-ratelimit-reset-requests: 8.64s
-x-ratelimit-reset-tokens: 6ms
-
    -

    Implementation Details

    Stream Completion

    The stream ends with:

    - -

    Usage Information

    Final message includes token usage:

    -
    "data":{
-   "id":"chatcmpl-AiT7GQk8zzYSC0Q8UT1pzyRzwxBCN",
-   "object":"chat.completion.chunk",
-   "created":1735161718,
-   "model":"gpt-4o-mini-2024-07-18",
-   "system_fingerprint":"fp_0aa8d3e20b",
-   "choices":[
-      
-   ],
-   "usage":{
-      "prompt_tokens":11,
-      "completion_tokens":18,
-      "total_tokens":29,
-      "prompt_tokens_details":{
-         "cached_tokens":0,
-         "audio_tokens":0
-      },
-      "completion_tokens_details":{
-         "reasoning_tokens":0,
-         "audio_tokens":0,
-         "accepted_prediction_tokens":0,
-         "rejected_prediction_tokens":0
-      }
-   }
-}
-
    -

    Conclusion

    SSE provides an elegant solution for real-time, server-to-client communications. Its simplicity, efficiency, and integration with existing infrastructure make it an excellent choice for many applications. While WebSockets remain valuable for bidirectional communication, SSE offers a more focused and often more appropriate solution for one-way data streaming scenarios.

    - - - - - - -

    - - #webdev - -

    - - - - - - - - - - -
    - - diff --git a/docs/posts/siyuan-privacy-first-self-hosted-personal-knowledge-management-software.html b/docs/posts/siyuan-privacy-first-self-hosted-personal-knowledge-management-software.html index 313480baf5e..cad8707b485 100644 --- a/docs/posts/siyuan-privacy-first-self-hosted-personal-knowledge-management-software.html +++ b/docs/posts/siyuan-privacy-first-self-hosted-personal-knowledge-management-software.html @@ -17,52 +17,418 @@ Original

    Siyuan: Privacy-first, self-hosted personal knowledge management software

    -
    -

    After many long years, I am trying to write a small web UI for a side project. It is going to be a single-page application that I would like to be served from my server written in Go. I was introduced to vite which gave me a hello world react app that gave me web assets like

    -
    1
    webapp/
    -

    Basic serve

    In order to serve that app from my Go server, I would initialize it to be a go package that exposes a web server that serves it. That means creating webapp/webapp.go in our vite project.

    -
    1
    package webapp
    -

    Now our react app is also a Go package which could be consumed from other packages like this

    -
    1
    webServer := webapp.New(webServerAddr)
    -

    Searching for a handler

    We would like to serve whatever is in our dist folder when someone hits our web server. So I searched if there were any such handlers and found two such functions in the standard library. So that we could register the handler in our routes ServeMux.

    -
    1
    func FileServer(root FileSystem) Handler
    -

    The difference between them is their arguments. FileSystem is part of the net/http library and I found this note about it in the comments:

    -
    1
    // This interface predates the [fs.FS] interface, which can be used instead:
    -

    So it is a deprecated interface in favor of a new interface. That leaves us with fs.FS.

    -

    It seems like fs.FS is an interface that represents a file system and is present in the io/fs package. I remember that they introduced the io package in the standard library to deprecate the ioutil package and this should have been a sweet addition that came with it?

    -

    Anyway, we will choose FileServerFS now.

    -
    1
    routes.Handle("/", http.FileServerFS(????))
    -

    A filesystem

    So we need a filesystem that we can give to our http.FileServerFS() method. I searched through the possible implementations of fs.FS interface and the first one that I was able to surface is os.DirFS.

    -

    It is a function that gives fs.FS based on the contents of a file system directory.

    -
    1
    routes.Handle("/", http.FileServerFS(os.DirFS("/home/vishnu/pers/gokakashi/webapp/dist")))
    -

    The problem with this approach is, that we will hit troubles when we ship our server to a machine on the internet. Now, someone has to take care of creating /home/vishnu/pers/gokakashi/webapp/dist on the server machine or point to a directory that contains our web app’s assets. That becomes messy, right?

    -

    What I would ideally want is to embed all my HTML, CSS, JS files inside our go server binary itself. That way we ship only the binary and it will be able to serve the web app.

    -

    Baking

    I have heard of the embed package which could be used for embedding files inside the go binary. There is embed.FS which satisfied fs.FS interface. So we could probably use it to bake in our assets.

    -
    1
    //go:embed dist
    -

    We make use of it like

    -
    1
    routes.Handle("/", http.FileServerFS(WebAssets))
    -

    This renders:

    -

    (not quite, what we want) Instead of embedding what is inside dist folder, we have embedded the dist folder itself. To inspect what got embedded, we could use the go list command like this:

    -
    1
    $ go list -f '{{.EmbedFiles}}' .
    -

    I tried doing //go:embed dist/* instead of //go:embed dist, that didn’t help as well because (from Go docs)

    -
    -

    The difference is that ‘image/*’ embeds ‘image/.tempfile’ while ‘image’ does not. Neither embeds ‘image/dir/.tempfile’.

    -
    -

    If I used //go:embed dist/index.html, then the file seems to be still embedded with the dist folder.

    -
    1
    $ go list -f '{{.EmbedFiles}}' .
    -

    I was wondering if there is a way to embed the contents of dist at the root of the FS instead of the FS directory. There seems to be no provision in the embed package to do it. Because it operates at a package level, and it would force us to declare dist as a separate go package and would force us to remove dist folder from .gitignore. That would be messy.

    -

    Traverse

    We know that we can’t change the root of the file system, but what if we traverse to a folder and get an fs.FS representation that projects the selected directory as the root? I suspected that there might be method to help with that in the io/fs package.

    -

    yep, look what I discovered:

    -
    1
    // Sub returns an FS corresponding to the subtree rooted at fsys's dir.
    -

    Now, we just make use of it like this

    -
    1
    //go:embed dist
    -

    And boom!

    -

    Screenshot from 2024-12-21 12-48-57

    -

    Compile time

    The //go:embed dist directive is evaluated during compile time. So when you run go build, the compiler looks for a dist folder and bakes it in the binary.

    -

    Let us say we miss generating the dist folder (maybe we failed running npm run build), that would lead to a compile-time error

    -
    1
    $ go build
    -

    This way we get the guarantee that no one is able to build our Go app without building the frontend that is intended to be embedded in it.

    - -
    +

    +SiYuan +

    +

    +中文 | 日本語 +

    +
    + + +
    + +

    SiYuan is a privacy-first personal knowledge management system, support fine-grained block-level reference and Markdown +WYSIWYG.

    +

    Welcome to SiYuan English Discussion Forum to learn more.

    +

    feature0.png

    +

    feature51.png

    + +

    Most features are free, even for commercial use.

    +
      +
    • Content block +
        +
      • Block-level reference and two-way links
        • +
      • Custom attributes
        • +
      • SQL query embed
        • +
      • Protocol siyuan://
        • +
      +
      • +
    • Editor +
        +
      • Block-style
        • +
      • Markdown WYSIWYG
        • +
      • List outline
        • +
      • Block zoom-in
        • +
      • Million-word large document editing
        • +
      • Mathematical formulas, charts, flowcharts, Gantt charts, timing charts, staffs, etc.
        • +
      • Web clipping
        • +
      • PDF Annotation link
        • +
      +
      • +
    • Export +
        +
      • Block ref and embed
        • +
      • Standard Markdown with assets
        • +
      • PDF, Word and HTML
        • +
      • Copy to WeChat MP, Zhihu and Yuque
        • +
      +
      • +
    • Database +
        +
      • Table view
        • +
      +
      • +
    • Flashcard spaced repetition
      • +
    • AI writing and Q/A chat via OpenAI API
      • +
    • Tesseract OCR
      • +
    • Multi-tab, drag and drop to split screen
      • +
    • Template snippet
      • +
    • JavaScript/CSS snippet
      • +
    • Android/iOS/HarmonyOS App
      • +
    • Docker deployment
      • +
    • API
      • +
    • Community marketplace
      • +
    +

    Some features are only available to paid members, for more details please refer to Pricing.

    +

    🏗️ Architecture and Ecosystem

    +

    SiYuan Arch

    + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    ProjectDescriptionForksStars
    luteEditor engineGitHub forksGitHub Repo stars
    chromeChrome/Edge extensionGitHub forksGitHub Repo stars
    bazaarCommunity marketplaceGitHub forksGitHub Repo stars
    dejavuData repoGitHub forksGitHub Repo stars
    petalPlugin APIGitHub forksGitHub Repo stars
    androidAndroid AppGitHub forksGitHub Repo stars
    iosiOS AppGitHub forksGitHub Repo stars
    harmonyHarmonyOS AppGitHub forksGitHub Repo stars
    riffSpaced repetitionGitHub forksGitHub Repo stars
     + + + + + + Star History Chart + + + + + +

    It is recommended to give priority to installing through the application market on the desktop and mobile, so that you can upgrade the version with one click in the future.

    + +

    Mobile:

    + +

    Desktop:

    + + + + +
    +Docker Deployment + +

    The easiest way to serve SiYuan on a server is to deploy it through Docker.

    + + +

    The overall program is located under /opt/siyuan/, which is basically the structure under the resources folder of the Electron installation package:

    +
      +
    • appearance: icon, theme, languages
      • +
    • guide: user guide document
      • +
    • stage: interface and static resources
      • +
    • kernel: kernel program
      • +
    + +

    The entry point is set when building the Docker image: ENTRYPOINT ["/opt/siyuan/entrypoint.sh"]. This script allows changing the PUID and PGID of the user that will run inside the container. This is especially relevant to solve permission issues when mounting directories from the host. The PUID (User ID) and PGID (Group ID) can be passed as environment variables, making it easier to ensure correct permissions when accessing host-mounted directories.

    +

    Use the following parameters when running the container with docker run b3log/siyuan:

    +
      +
    • --workspace: Specifies the workspace folder path, mounted to the container via -v on the host
      • +
    • --accessAuthCode: Specifies the access authorization code
      • +
    +

    More parameters can be found using --help. Here’s an example of a startup command with the new environment variables:

    +
    docker run -d \
+  -v workspace_dir_host:workspace_dir_container \
+  -p 6806:6806 \
+  -e PUID=1001 -e PGID=1002 \
+  b3log/siyuan \
+  --workspace=workspace_dir_container \
+  --accessAuthCode=xxx
    +
      +
    • PUID: Custom user ID (optional, defaults to 1000 if not provided)
      • +
    • PGID: Custom group ID (optional, defaults to 1000 if not provided)
      • +
    • workspace_dir_host: The workspace folder path on the host
      • +
    • workspace_dir_container: The path of the workspace folder in the container, as specified in --workspace
      • +
    • accessAuthCode: Access authorization code (please be sure to modify, otherwise anyone can access your data)
      • +
    +

    To simplify things, it is recommended to configure the workspace folder path to be consistent on the host and container, such as having both workspace_dir_host and workspace_dir_container configured as /siyuan/workspace. The corresponding startup command would be:

    +
    docker run -d \
+  -v /siyuan/workspace:/siyuan/workspace \
+  -p 6806:6806 \
+  -e PUID=1001 -e PGID=1002 \
+  b3log/siyuan \
+  --workspace=/siyuan/workspace/ \
+  --accessAuthCode=xxx
    + +

    For users running Siyuan with Docker Compose, the environment variables PUID and PGID can be passed to customize the user and group IDs. Here's an example of a Docker Compose configuration:

    +
    version: "3.9"
+services:
+  main:
+    image: b3log/siyuan
+    command: ['--workspace=/siyuan/workspace/', '--accessAuthCode=${AuthCode}']
+    ports:
+      - 6806:6806
+    volumes:
+      - /siyuan/workspace:/siyuan/workspace
+    restart: unless-stopped
+    environment:
+      # A list of time zone identifiers can be found at https://en.wikipedia.org/wiki/List_of_tz_database_time_zones
+      - TZ=${YOUR_TIME_ZONE}
+      - PUID=${YOUR_USER_PUID}  # Customize user ID
+      - PGID=${YOUR_USER_PGID}  # Customize group ID
    +

    In this setup:

    +
      +
    • PUID and PGID are set dynamically and passed to the container
      • +
    • If these variables are not provided, the default 1000 will be used
      • +
    +

    By specifying PUID and PGID in the environment, you avoid the need to explicitly set the user directive (user: '1000:1000') in the compose file. The container will dynamically adjust the user and group based on these environment variables at startup.

    + +

    In the image, the entrypoint.sh script ensures the creation of the siyuan user and group with the specified PUID and PGID. Therefore, when the host creates a workspace folder, pay attention to setting the user and group ownership of the folder to match the PUID and PGID you plan to use. For example:

    +
    chown -R 1001:1002 /siyuan/workspace
    +

    If you use custom PUID and PGID values, the entrypoint script will ensure that the correct user and group are created inside the container, and ownership of mounted volumes will be adjusted accordingly. There’s no need to manually pass -u in docker run or docker-compose as the environment variables will handle the customization.

    + +

    Use NGINX reverse proxy to hide port 6806, please note:

    +
      +
    • Configure WebSocket reverse proxy /ws
      • +
    + +
      +
    • Be sure to confirm the correctness of the mounted volume, otherwise the data will be lost after the container is deleted
      • +
    • Do not use URL rewriting for redirection, otherwise there may be problems with authentication, it is recommended to configure a reverse proxy
      • +
    • If you encounter permission issues, verify that the PUID and PGID environment variables match the ownership of the mounted directories on your host system
      • +
    + +
      +
    • Does not support desktop and mobile application connections, only supports use on browsers
      • +
    • Export to PDF, HTML and Word formats is not supported
      • +
    • Import Markdown file is not supported
      • +
    +
    + +
    +Unraid Deployment +

    Note: First run chown -R 1000:1000 /mnt/user/appdata/siyuan in the terminal

    +

    Template reference:

    +
    Web UI: 6806
+Container Port: 6806
+Container Path: /home/siyuan
+Host path: /mnt/user/appdata/siyuan
+PUID: 1000
+PGID: 1000
+Publish parameters: --accessAuthCode=******(Access authorization code)
+
    +
    + +

    We release insider preview before major updates, please visit https://github.com/siyuan-note/insider.

    + + +

    🛠️ Development Guide

    +

    See Development Guide.

    + +

    How does SiYuan store data?

    +

    The data is saved in the workspace folder, in the workspace data folder:

    +
      +
    • assets is used to save all inserted assets
      • +
    • emojis is used to save emoji images
      • +
    • snippets is used to save code snippets
      • +
    • storage is used to save query conditions, layouts and flashcards, etc.
      • +
    • templates is used to save template snippets
      • +
    • widgets is used to save widgets
      • +
    • plugins is used to save plugins
      • +
    • public is used to save public data
      • +
    • The rest of the folders are the notebook folders created by the user, files with the suffix of .sy in the notebook folder are used to save the document data, and the data format is JSON
      • +
    +

    Does it support data synchronization through a third-party sync disk?

    +

    Data synchronization through third-party synchronization disks is not supported, otherwise data may be corrupted.

    +

    Although it does not support third-party sync disks, it supports connect with third-party cloud storage (Member's privileges).

    +

    In addition, you can also consider manually exporting and importing data to achieve data synchronization:

    +
      +
    • Desktop: Settings - Export - Export Data / Import Data
      • +
    • Mobile: Right column - About - Export Data / Import Data
      • +
    + +

    SiYuan is completely open source, and contributions are welcome:

    + +

    For more details, please refer to Development Guide.

    +

    How to upgrade to a new version?

    +
      +
    • If installed via app store, please update via app store
      • +
    • If it is installed through the installation package on the desktop, you can open the option of Settings - About - Automatically download update installation package, so that SiYuan will automatically download The latest version of the installation package and prompts to install
      • +
    • If it is installed by manual installation package, please download the installation package again to install
      • +
    +

    You can Check update in Settings - About - Current Version, or pay attention to Official Download or GitHub Releases to get the new version.

    +

    What if some blocks (such as paragraph blocks in list items) cannot find the block icon?

    +

    The first sub-block under the list item is the block icon omitted. You can move the cursor into this block and trigger its block menu with Ctrl+/ .

    +

    What should I do if the data repo key is lost?

    +
      +
    • +

      If the data repo key is correctly initialized on multiple devices before, the key is the same on all devices and can be set in Settings - About - Data repo key - Copy key string retrieve

      +
      • +
    • +

      If it has not been configured correctly before (for example, the keys on multiple devices are inconsistent) or all devices are unavailable and the key string cannot be obtained, you can reset the key by following the steps below:

      +
        +
      1. Manually back up the data, you can use Export Data or directly copy the workspace/data/ folder on the file system
        2. +
      2. Settings - About - Data rep key - Reset data repo
        3. +
      3. Reinitialize the data repo key. After initializing the key on one device, other devices import the key
        4. +
      4. The cloud uses the new synchronization directory, the old synchronization directory is no longer available and can be deleted
        5. +
      5. The existing cloud snapshots are no longer available and can be deleted
        6. +
      +
      • +
    + +

    Most features are free, even for commercial use.

    +

    Member's privileges can only be used after payment, please refer to Pricing.

    + +

    The birth of SiYuan is inseparable from many open source projects and contributors, please refer to the project source code kernel/go.mod, app/package.json and project homepage.

    +

    The growth of SiYuan is inseparable from user feedback and promotion, thank you for everyone's help to SiYuan ❤️

    + +

    Welcome to join us and contribute code to SiYuan together.

    + + + +
    diff --git a/docs/posts/the-swedish-cabin-on-the-frontline-of-a-possible-hybrid-war.html b/docs/posts/the-swedish-cabin-on-the-frontline-of-a-possible-hybrid-war.html index b89aae9e055..f426f8e931b 100644 --- a/docs/posts/the-swedish-cabin-on-the-frontline-of-a-possible-hybrid-war.html +++ b/docs/posts/the-swedish-cabin-on-the-frontline-of-a-possible-hybrid-war.html @@ -17,6 +17,6 @@ Original

    The Swedish cabin on the frontline of a possible hybrid war

    -

    At the end of an unmarked path on a tiny island at the edge of Stockholm’s extensive Baltic Sea archipelago lies an inconspicuous little wooden cabin, painted a deep shade of red. Water gently laps the snow-dusted rocks, and the smell of pine fills the air.

    The site offers few clues to the geopolitical drama that has gripped Scandinavia in recent months, driven by accusations of infrastructure sabotage. But in fact the cabin houses a key cog in Europe’s digital connectivity, and a point of vulnerability in a potential hybrid war: a datacentre that amplifies the signal from a 1,615-mile fibre-optic cable running from northern Sweden to Berlin.

    Last month, two nearby fibre-optic cables were severed, prompting a continuing investigation by Swedish authorities. Western intelligence officials from multiple countries have said they are confident a Chinese ship caused the cuts after leaving the Russian port of Ust-Luga, though views differ on whether the cuts were accidental or potentially deliberate.

    View inside the datacentre, with wiring running around at the edge of the ceiling above grey metal doors housing the computer systems
    Inside, the datacentre has ‘the normal stuff’ such as alarms, CCTV and access control. Photograph: Josefine Stenersen/The Guardian

    Since Russia’s invasion of Ukraine in 2022, Sweden has experienced a rise in hybrid warfare – attacks on an adversary using methods other than traditional military action – blamed on pro-Russia groups. With governments in northern Europe on high alert over hybrid Russian activity, the Guardian was given exclusive access to the Stockholm datacentre site.

    Daniel Aldstam, the chief security officer at GlobalConnect, which transports 50% of the internet capacity of the Nordics and runs the centre, described the approach to its location and ordinary outward appearance as “security through obscurity”.

    “Essentially you have two different approaches,” he said. “Either you put a lot of fences around it which make it obvious that there is something critical, or you do it like we have done it here and try to keep things a little more discreet. But of course we have the normal stuff in terms of alarms, CCTV, access control and all of that.” Inside, cages full of equipment emit blinking lights and different-coloured cables line the ceiling.

    After a recent suspected sabotage incident, the Polish prime minister, Donald Tusk, proposed a “navy policing” initiative involving joint military patrols by countries around the Baltic.

    Map of where cables were severed

    Travelling from Stockholm by helicopter over the archipelago, formed of 30,000 islands, rocks and skerries, it is clear to see how challenging the coastline is to protect. But its vastness also suggests how the “security through obscurity” approach could be effective – at least up to a point. Maps that show where all the undersea cables are laid are publicly available.

    “We have hundreds of thousands of kilometres of fibre. How do you physically protect it? You can’t,” said Aldstam. “What is important here is the redundancy [using multiple cables offering alternative routes if one is cut off]. You need to have more fibres.”

    With infrastructure seen as being particularly vulnerable to hybrid warfare, there are signs of tweaks to the “obscurity” approach, reflecting the fraught times.

    (From left) GlobalConnect’s Pär Jansson, the senior vice-president; Patrik Gylesjö, the project manager; and the chief security officer Daniel Aldstam.
    (From left) GlobalConnect’s Pär Jansson, the senior vice-president; Patrik Gylesjö, the project manager; and the chief security officer Daniel Aldstam. Photograph: Josefine Stenersen/The Guardian

    GlobalConnect is in the process of setting up a bigger and more modern-looking datacentre nearby, which although still unmarked and painted a similar shade of red, is more obviously a building doing an important function. Inside it has its own diesel-powered backup generator to ensure it could continue running if electricity was cut off.

    The vulnerability to sabotage of undersea cables and other critical infrastructure – particularly in the relatively shallow and busy Baltic – has come into sharp focus since Russia’s full-scale invasion of Ukraine.

    Map of Europe’s submarine data cables

    In September 2022, the Nord Stream pipeline, which carried natural gas from Russia to Germany, was blown up. Initially, many assumed Russia was to blame. However, in August this year, German media reported that German authorities had issued an arrest warrant for a Ukrainian man on suspicion of being part of a team that planted explosive devices on the pipeline. Both sides in the war in Ukraine have denied responsibility and blame each other for the attack.

    Nato, which has established a dedicated centre for undersea security, has warned that the security of nearly 1 billion people across Europe and North America is at risk of hybrid warfare by the alliance’s adversaries, due to vulnerabilities in windfarm, pipeline and power cable infrastructure. Earlier this month, the Nato secretary general, Mark Rutte, urged Europeans to “shift to a wartime mindset”.

    For all the warnings, undersea cables, which can lie on or be buried in the seabed, look surprisingly slight.

    “We call it a super mega cable, but it doesn’t sound super mega or look super mega,” said Patrik Gylesjö, who is responsible for overseeing the whole of GlobalConnect’s Sweden to Berlin cable project, which was completed earlier this year. “The name refers to its capacity rather than the size.”

    A man holds up a fibre-optic cable that has been severed to show its interior
    The 2cm fibre-optic cables can support 1bn simultaneous Netflix streams. Photograph: Josefine Stenersen/The Guardian

    Inside the cable, which is little more than 2cm in diameter, is a small section formed of 96 hair-thin fibre pairs – enough to support 1bn simultaneous Netflix streams, he said. The rest is made up of steel armouring and a waterproofing substance.

    It would take only an anchor from a relatively small ship to break the cable, said Gylesjö. “If you would like to break this cable or cut it, you would not need a super-big tool. It’s quite fragile.”

    Making it stronger, he added, would make it heaver, more expensive and “more complicated to deploy”.

    Accidental breaks in undersea cables are incredibly rare. “It’s very rare that damage happens in general,” said Gylesjö. “Very rare. During our time as an operator of sea cables [more than 20 years] I think it has happened two to three times maximum.”

    +

    At the end of an unmarked path on a tiny island at the edge of Stockholm’s extensive Baltic Sea archipelago lies an inconspicuous little wooden cabin, painted a deep shade of red. Water gently laps the snow-dusted rocks, and the smell of pine fills the air.

    The site offers few clues to the geopolitical drama that has gripped Scandinavia in recent months, driven by accusations of infrastructure sabotage. But in fact the cabin houses a key cog in Europe’s digital connectivity, and a point of vulnerability in a potential hybrid war: a datacentre that amplifies the signal from a 1,615-mile fibre-optic cable running from northern Sweden to Berlin.

    Last month, two nearby fibre-optic cables were severed, prompting a continuing investigation by Swedish authorities. Western intelligence officials from multiple countries have said they are confident a Chinese ship caused the cuts after leaving the Russian port of Ust-Luga, though views differ on whether the cuts were accidental or potentially deliberate.

    View inside the datacentre, with wiring running around at the edge of the ceiling above grey metal doors housing the computer systems
    Inside, the datacentre has ‘the normal stuff’ such as alarms, CCTV and access control. Photograph: Josefine Stenersen/The Guardian

    Since Russia’s invasion of Ukraine in 2022, Sweden has experienced a rise in hybrid warfare – attacks on an adversary using methods other than traditional military action – blamed on pro-Russia groups. With governments in northern Europe on high alert over hybrid Russian activity, the Guardian was given exclusive access to the Stockholm datacentre site.

    Daniel Aldstam, the chief security officer at GlobalConnect, which transports 50% of the internet capacity of the Nordics and runs the centre, described the approach to its location and ordinary outward appearance as “security through obscurity”.

    “Essentially you have two different approaches,” he said. “Either you put a lot of fences around it which make it obvious that there is something critical, or you do it like we have done it here and try to keep things a little more discreet. But of course we have the normal stuff in terms of alarms, CCTV, access control and all of that.” Inside, cages full of equipment emit blinking lights and different-coloured cables line the ceiling.

    After a recent suspected sabotage incident, the Polish prime minister, Donald Tusk, proposed a “navy policing” initiative involving joint military patrols by countries around the Baltic.

    Map of where cables were severed

    Travelling from Stockholm by helicopter over the archipelago, formed of 30,000 islands, rocks and skerries, it is clear to see how challenging the coastline is to protect. But its vastness also suggests how the “security through obscurity” approach could be effective – at least up to a point. Maps that show where all the undersea cables are laid are publicly available.

    “We have hundreds of thousands of kilometres of fibre. How do you physically protect it? You can’t,” said Aldstam. “What is important here is the redundancy [using multiple cables offering alternative routes if one is cut off]. You need to have more fibres.”

    With infrastructure seen as being particularly vulnerable to hybrid warfare, there are signs of tweaks to the “obscurity” approach, reflecting the fraught times.

    (From left) GlobalConnect’s Pär Jansson, the senior vice-president; Patrik Gylesjö, the project manager; and the chief security officer Daniel Aldstam.
    (From left) GlobalConnect’s Pär Jansson, the senior vice-president; Patrik Gylesjö, the project manager; and the chief security officer Daniel Aldstam. Photograph: Josefine Stenersen/The Guardian

    GlobalConnect is in the process of setting up a bigger and more modern-looking datacentre nearby, which although still unmarked and painted a similar shade of red, is more obviously a building doing an important function. Inside it has its own diesel-powered backup generator to ensure it could continue running if electricity was cut off.

    The vulnerability to sabotage of undersea cables and other critical infrastructure – particularly in the relatively shallow and busy Baltic – has come into sharp focus since Russia’s full-scale invasion of Ukraine.

    Map of Europe’s submarine data cables

    In September 2022, the Nord Stream pipeline, which carried natural gas from Russia to Germany, was blown up. Initially, many assumed Russia was to blame. However, in August this year, German media reported that German authorities had issued an arrest warrant for a Ukrainian man on suspicion of being part of a team that planted explosive devices on the pipeline. Both sides in the war in Ukraine have denied responsibility and blame each other for the attack.

    Nato, which has established a dedicated centre for undersea security, has warned that the security of nearly 1 billion people across Europe and North America is at risk of hybrid warfare by the alliance’s adversaries, due to vulnerabilities in windfarm, pipeline and power cable infrastructure. Earlier this month, the Nato secretary general, Mark Rutte, urged Europeans to “shift to a wartime mindset”.

    For all the warnings, undersea cables, which can lie on or be buried in the seabed, look surprisingly slight.

    “We call it a super mega cable, but it doesn’t sound super mega or look super mega,” said Patrik Gylesjö, who is responsible for overseeing the whole of GlobalConnect’s Sweden to Berlin cable project, which was completed earlier this year. “The name refers to its capacity rather than the size.”

    A man holds up a fibre-optic cable that has been severed to show its interior
    The 2cm fibre-optic cables can support 1bn simultaneous Netflix streams. Photograph: Josefine Stenersen/The Guardian

    Inside the cable, which is little more than 2cm in diameter, is a small section formed of 96 hair-thin fibre pairs – enough to support 1bn simultaneous Netflix streams, he said. The rest is made up of steel armouring and a waterproofing substance.

    It would take only an anchor from a relatively small ship to break the cable, said Gylesjö. “If you would like to break this cable or cut it, you would not need a super-big tool. It’s quite fragile.”

    Making it stronger, he added, would make it heaver, more expensive and “more complicated to deploy”.

    Accidental breaks in undersea cables are incredibly rare. “It’s very rare that damage happens in general,” said Gylesjö. “Very rare. During our time as an operator of sea cables [more than 20 years] I think it has happened two to three times maximum.”

    diff --git a/docs/posts/ugandan-runner-due-to-arrive-in-london-after-516-days-7-700-miles-on-the-road.html b/docs/posts/ugandan-runner-due-to-arrive-in-london-after-516-days-7-700-miles-on-the-road.html index 5ea4780977b..29413f4ca6f 100644 --- a/docs/posts/ugandan-runner-due-to-arrive-in-london-after-516-days-7-700-miles-on-the-road.html +++ b/docs/posts/ugandan-runner-due-to-arrive-in-london-after-516-days-7-700-miles-on-the-road.html @@ -17,6 +17,6 @@ Original

    Ugandan runner due to arrive in London after 516 days, 7,700 miles on the road

    -

    A Ugandan athlete who arrives in London this weekend after running 7,730 miles (12,440km) from South Africa to raise awareness about racism has revealed he suffered repeated abuse on reaching Europe.

    Deo Kato set off from Cape Town in July 2023, running steadily north on a 516-day odyssey that has seen him jailed for weeks, laid low with serious illness and having to pass through war zones.

    The epic run was conceived by the London-based Kato to highlight the history of human migration and the discrimination faced by many black Africans, a message underlined by the fact he endured daily racism from police and passersby in parts of Europe.

    After climbing the equivalent height of 11 Mount Everests during the journey, Kato is due to reach central London on Sunday where he will be joined by hundreds of runners outside Downing Street before completing his route in Hammersmith, west London.

    A group of African children and a man run along a road
    Local children joined Kato on his run on their way to school. Photograph: Deo Kato

    Speaking to the Guardian this week after passing through an overcast Lille, in France, the Ugandan-born runner said that despite some acute lows the overall experience had renewed his faith in humanity. Highlights included a stretch along the Kalahari Highway in Botswana where he was joined by a 15-year-old boy who, Kato said, reminded him of when he was a teenager.

    “He was multilingual, speaking three languages, including English. He had spent time in England but moved back to Botswana due to family challenges. We ran together briefly, but it was a moment that warmed my heart.”

    Another affirming moment, this time 1,800 miles farther north in Kenya during January, involved a group of children who spontaneously joined Kato for 5 miles as they headed to school. “They wanted to continue running with me,” he said.

    On other occasions, however, he almost packed it in. In Uganda, his one-man support crew resigned, leaving him without a support vehicle or help at a time when his funding for the run was almost exhausted. To compound matters, all routes ahead involved either conflict or extreme risk.

    Deo Kato running the London Marathon 2021.
    Kato says his run is to challenge the racist idea that people should ‘go back to where they come from’. Photograph: Alamy

    “As I looked forward, I noticed conflicts all around me in places such as the eastern Democratic Republic of the Congo, Sudan and northern Ethiopia,” said Kato. “Logistically, it felt that there was no possible way of continuing the journey through Africa.”

    Another low point arrived more than 5,000 miles later when Kato experienced the racism other Africans have faced in Europe.

    “The other time I felt like packing it in was in Croatia because I genuinely felt treated as an illegal immigrant. I didn’t feel welcomed or that I belonged in their society.

    “The police stopped me at least four times a day. Sometimes, I caught locals taking photos of me and reporting me to the police,” he said.

    Kato and his partner, Alice Light.
    Kato and his partner, Alice Light, who has described the 18-month trip as an ‘unimaginable rollercoaster of highs and lows’. Photograph: Deo Kato

    “This experience, coupled with everything I was processing from my journey in Africa and other personal challenges, made it intensely difficult to keep moving forward.”

    Kato wanted his journey to draw attention to the earliest migration of humans from Africa and challenge the racist notion that people should “go back to where they come from”. Viewed as a whole, he said the run had underlined the positive aspects of migration and its potential to “create a more culturally connected and enriched global society”.

    His experiences also led him to believe that humanity will prevail over prejudice. “I think that in the future, we will create a world free from racial discrimination,” he said.

    “Although it won’t happen in my lifetime, I believe that my efforts and those of others who are dedicated to this cause are laying the foundations for the next generation to build upon.”

    However, he admitted it had also reinforced concerns over the “fortress Europe” approach that the EU is pursuing to prevent migrants from Africa moving north.

    “The global north has long-established systems deliberately designed to restrict and criminalise individuals from the global south, particularly Africans.”

    Kato’s partner, Alice Light, said the last 18 months had highlighted the best and worst of humanity. “It’s been an unimaginable rollercoaster of highs and lows, of beauty, joy and heartbreak,” she said.

    She said the couple had no plans for Christmas. “It has been too unpredictable to make plans but I now know that rest is coming and am grateful for that. I feel immensely proud and blessed to have been on this journey with Deo.”

    Kato set off from the Long March to Freedom monument in Cape Town, which commemorates the anti-apartheid struggle, choosing the eastern route through Africa because he wanted to pass through the Ugandan town of Nakulabye, where he grew up, to meet family members he had not seen for years.

    +

    A Ugandan athlete who arrives in London this weekend after running 7,730 miles (12,440km) from South Africa to raise awareness about racism has revealed he suffered repeated abuse on reaching Europe.

    Deo Kato set off from Cape Town in July 2023, running steadily north on a 516-day odyssey that has seen him jailed for weeks, laid low with serious illness and having to pass through war zones.

    The epic run was conceived by the London-based Kato to highlight the history of human migration and the discrimination faced by many black Africans, a message underlined by the fact he endured daily racism from police and passersby in parts of Europe.

    After climbing the equivalent height of 11 Mount Everests during the journey, Kato is due to reach central London on Sunday where he will be joined by hundreds of runners outside Downing Street before completing his route in Hammersmith, west London.

    A group of African children and a man run along a road
    Local children joined Kato on his run on their way to school. Photograph: Deo Kato

    Speaking to the Guardian this week after passing through an overcast Lille, in France, the Ugandan-born runner said that despite some acute lows the overall experience had renewed his faith in humanity. Highlights included a stretch along the Kalahari Highway in Botswana where he was joined by a 15-year-old boy who, Kato said, reminded him of when he was a teenager.

    “He was multilingual, speaking three languages, including English. He had spent time in England but moved back to Botswana due to family challenges. We ran together briefly, but it was a moment that warmed my heart.”

    Another affirming moment, this time 1,800 miles farther north in Kenya during January, involved a group of children who spontaneously joined Kato for 5 miles as they headed to school. “They wanted to continue running with me,” he said.

    On other occasions, however, he almost packed it in. In Uganda, his one-man support crew resigned, leaving him without a support vehicle or help at a time when his funding for the run was almost exhausted. To compound matters, all routes ahead involved either conflict or extreme risk.

    Deo Kato running the London Marathon 2021.
    Kato says his run is to challenge the racist idea that people should ‘go back to where they come from’. Photograph: Alamy

    “As I looked forward, I noticed conflicts all around me in places such as the eastern Democratic Republic of the Congo, Sudan and northern Ethiopia,” said Kato. “Logistically, it felt that there was no possible way of continuing the journey through Africa.”

    Another low point arrived more than 5,000 miles later when Kato experienced the racism other Africans have faced in Europe.

    “The other time I felt like packing it in was in Croatia because I genuinely felt treated as an illegal immigrant. I didn’t feel welcomed or that I belonged in their society.

    “The police stopped me at least four times a day. Sometimes, I caught locals taking photos of me and reporting me to the police,” he said.

    Kato and his partner, Alice Light.
    Kato and his partner, Alice Light, who has described the 18-month trip as an ‘unimaginable rollercoaster of highs and lows’. Photograph: Deo Kato

    “This experience, coupled with everything I was processing from my journey in Africa and other personal challenges, made it intensely difficult to keep moving forward.”

    Kato wanted his journey to draw attention to the earliest migration of humans from Africa and challenge the racist notion that people should “go back to where they come from”. Viewed as a whole, he said the run had underlined the positive aspects of migration and its potential to “create a more culturally connected and enriched global society”.

    His experiences also led him to believe that humanity will prevail over prejudice. “I think that in the future, we will create a world free from racial discrimination,” he said.

    “Although it won’t happen in my lifetime, I believe that my efforts and those of others who are dedicated to this cause are laying the foundations for the next generation to build upon.”

    However, he admitted it had also reinforced concerns over the “fortress Europe” approach that the EU is pursuing to prevent migrants from Africa moving north.

    “The global north has long-established systems deliberately designed to restrict and criminalise individuals from the global south, particularly Africans.”

    Kato’s partner, Alice Light, said the last 18 months had highlighted the best and worst of humanity. “It’s been an unimaginable rollercoaster of highs and lows, of beauty, joy and heartbreak,” she said.

    She said the couple had no plans for Christmas. “It has been too unpredictable to make plans but I now know that rest is coming and am grateful for that. I feel immensely proud and blessed to have been on this journey with Deo.”

    Kato set off from the Long March to Freedom monument in Cape Town, which commemorates the anti-apartheid struggle, choosing the eastern route through Africa because he wanted to pass through the Ugandan town of Nakulabye, where he grew up, to meet family members he had not seen for years.

    diff --git a/docs/posts/weeknotes-28.html b/docs/posts/weeknotes-28.html index 0ef551de3a4..7c67e85887d 100644 --- a/docs/posts/weeknotes-28.html +++ b/docs/posts/weeknotes-28.html @@ -68,7 +68,7 @@

    Weeknotes #28

  • i feel like i'm really clear on what i want this year without a big chunk of planning, but i also want conflicting things e.g. fancy sleeve tattoo vs. saving a big emergency fund.
  • since i guess i won't write another thing on here weeknotes wise till after xmas i hope you all have a fantastic one if you celebrate, and a great day otherwise too! happy solstice for tomorrow, even if you don't celebrate it too (brighter days are coming, northern hemisphere! the hateful sun is going back in her cage, southern hemisphere!) xo
    • -

    Last updated 5 days, 17 hours ago

    +

    Last updated 5 days, 18 hours ago

    If you liked this post, please message, email, or follow me online, check out my work in progress, share this post or subscribe to my posts by RSS!