-
Notifications
You must be signed in to change notification settings - Fork 198
/
USAGE-SQFSTAR-4.6
812 lines (621 loc) · 33.3 KB
/
USAGE-SQFSTAR-4.6
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
SQFSTAR - A tool to create a Squashfs filesystem from a TAR archive
Sqfstar will read an uncompressed TAR archive from standard in, and it will
create a Squashfs filesystem from it. If a TAR archive is compressed it
should be piped through a decompressor utility and then input into Sqfstar.
Sqfstar supports V7, ustar, bsdtar (libarchive), GNU tar and PAX extensions.
Sparse file extensions are supported, including the "old GNU format, type S",
and PAX formats, Versions 0.0, 0.1 and the current 1.0.
Sqfstar supports extended attributes, and recognises the SCHILY xattr
PAX extension (used by GNU tar), and the LIBARCHIVE xattr PAX extension
(used by bsdtar).
The Sqfstar usage info is:
SYNTAX:sqfstar [OPTIONS] FILESYSTEM [list of exclude dirs/files]
Filesystem compression options:
-b <block_size> set data block to <block_size>. Default 128 Kbytes.
Optionally a suffix of K or M can be given to specify
Kbytes or Mbytes respectively
-comp <comp> select <comp> compression
Compressors available:
gzip (default)
lzo
lz4
xz
zstd
-noI do not compress inode table
-noId do not compress the uid/gid table (implied by -noI)
-noD do not compress data blocks
-noF do not compress fragment blocks
-noX do not compress extended attributes
-no-compression do not compress any of the data or metadata. This is
equivalent to specifying -noI -noD -noF and -noX
Filesystem build options:
-reproducible build filesystems that are reproducible (default)
-not-reproducible build filesystems that are not reproducible
-mkfs-time <time> set filesystem creation timestamp to <time>. <time> can
be an unsigned 32-bit int indicating seconds since the
epoch (1970-01-01) or a string value which is passed to
the "date" command to parse. Any string value which the
date command recognises can be used such as "now",
"last week", or "Wed Feb 15 21:02:39 GMT 2023"
-all-time <time> set all file timestamps to <time>. <time> can be an
unsigned 32-bit int indicating seconds since the epoch
(1970-01-01) or a string value which is passed to the
"date" command to parse. Any string value which the date
command recognises can be used such as "now", "last
week", or "Wed Feb 15 21:02:39 GMT 2023"
-root-time <time> set root directory time to <time>. <time> can be an
unsigned 32-bit int indicating seconds since the epoch
(1970-01-01) or a string value which is passed to the
"date" command to parse. Any string value which the date
command recognises can be used such as "now", "last
week", or "Wed Feb 15 21:02:39 GMT 2023"
-root-mode <mode> set root directory permissions to octal <mode>
-root-uid <value> set root directory owner to specified <value>,
<value> can be either an integer uid or user name
-root-gid <value> set root directory group to specified <value>,
<value> can be either an integer gid or group name
-all-root make all files owned by root
-force-uid <value> set all file uids to specified <value>, <value> can be
either an integer uid or user name
-force-gid <value> set all file gids to specified <value>, <value> can be
either an integer gid or group name
-default-mode <mode> tar files often do not store permissions for
intermediate directories. This option sets the default
directory permissions to octal <mode>, rather than 0755.
This also sets the root inode mode
-default-uid <uid> tar files often do not store uids for intermediate
directories. This option sets the default directory
owner to <uid>, rather than the user running Sqfstar.
This also sets the root inode uid
-default-gid <gid> tar files often do not store gids for intermediate
directories. This option sets the default directory
group to <gid>, rather than the group of the user
running Sqfstar. This also sets the root inode gid
-pseudo-override make pseudo file uids and gids override -all-root,
-force-uid and -force-gid options
-exports make the filesystem exportable via NFS
-no-sparse do not detect sparse files
-no-fragments do not use fragments
-no-tailends do not pack tail ends into fragments
-no-duplicates do not perform duplicate checking
-no-hardlinks do not hardlink files, instead store duplicates
Filesystem filter options:
-p <pseudo-definition> add pseudo file definition. The definition should
be quoted
-pf <pseudo-file> add list of pseudo file definitions. Pseudo file
definitions in pseudo-files should not be quoted
-ef <exclude_file> list of exclude dirs/files. One per line
-regex allow POSIX regular expressions to be used in exclude
dirs/files
-ignore-zeros allow tar files to be concatenated together and fed to
Sqfstar. Normally a tarfile has two consecutive 512
byte blocks filled with zeros which means EOF and
Sqfstar will stop reading after the first tar file on
encountering them. This option makes Sqfstar ignore the
zero filled blocks
Filesystem extended attribute (xattrs) options:
-no-xattrs do not store extended attributes
-xattrs store extended attributes (default)
-xattrs-exclude <regex> exclude any xattr names matching <regex>. <regex> is a
POSIX regular expression, e.g. -xattrs-exclude '^user.'
excludes xattrs from the user namespace
-xattrs-include <regex> include any xattr names matching <regex>. <regex> is a
POSIX regular expression, e.g. -xattrs-include '^user.'
includes xattrs from the user namespace
-xattrs-add <name=val> add the xattr <name> with <val> to files. If an
user xattr it will be added to regular files and
directories (see man 7 xattr). Otherwise it will be
added to all files. <val> by default will be treated as
binary (i.e. an uninterpreted byte sequence), but it can
be prefixed with 0s, where it will be treated as base64
encoded, or prefixed with 0x, where val will be treated
as hexidecimal. Additionally it can be prefixed with
0t where this encoding is similar to binary encoding,
except backslashes are specially treated, and a
backslash followed by 3 octal digits can be used to
encode any ASCII character, which obviously can be used
to encode control codes. The option can be repeated
multiple times to add multiple xattrs
Sqfstar runtime options:
-version print version, licence and copyright message
-force force Sqfstar to write to block device or file
-exit-on-error treat normally ignored errors as fatal
-quiet no verbose output
-info print files written to filesystem
-no-progress do not display the progress bar
-progress display progress bar when using the -info option
-percentage display a percentage rather than the full progress bar.
Can be used with dialog --gauge etc.
-throttle <percentage> throttle the I/O input rate by the given percentage.
This can be used to reduce the I/O and CPU consumption
of Sqfstar
-limit <percentage> limit the I/O input rate to the given percentage.
This can be used to reduce the I/O and CPU consumption
of Sqfstar (alternative to -throttle)
-processors <number> use <number> processors. By default will use number of
processors available
-mem <size> use <size> physical memory for caches. Use K, M or G to
specify Kbytes, Mbytes or Gbytes respectively
-mem-percent <percent> use <percent> physical memory for caches. Default 25%
-mem-default print default memory usage in Mbytes
Expert options (these may make the filesystem unmountable):
-nopad do not pad filesystem to a multiple of 4K
-offset <offset> skip <offset> bytes at the beginning of FILESYSTEM.
Optionally a suffix of K, M or G can be given to specify
Kbytes, Mbytes or Gbytes respectively.
Default 0 bytes
-o <offset> synonym for -offset
Miscellaneous options:
-fstime <time> alternative name for mkfs-time
-root-owned alternative name for -all-root
-noInodeCompression alternative name for -noI
-noIdTableCompression alternative name for -noId
-noDataCompression alternative name for -noD
-noFragmentCompression alternative name for -noF
-noXattrCompression alternative name for -noX
-help output this options text to stdout
-h output this options text to stdout
-Xhelp print compressor options for selected compressor
Pseudo file definition format:
"filename d mode uid gid" create a directory
"filename m mode uid gid" modify filename
"filename b mode uid gid major minor" create a block device
"filename c mode uid gid major minor" create a character device
"filename f mode uid gid command" create file from stdout of command
"filename s mode uid gid symlink" create a symbolic link
"filename i mode uid gid [s|f]" create a socket (s) or FIFO (f)
"filename x name=val" create an extended attribute
"filename l linkname" create a hard-link to linkname
"filename L pseudo_filename" same, but link to pseudo file
"filename D time mode uid gid" create a directory with timestamp time
"filename M time mode uid gid" modify a file with timestamp time
"filename B time mode uid gid major minor"
create block device with timestamp time
"filename C time mode uid gid major minor"
create char device with timestamp time
"filename F time mode uid gid command" create file with timestamp time
"filename S time mode uid gid symlink" create symlink with timestamp time
"filename I time mode uid gid [s|f]" create socket/fifo with timestamp time
Files and directories in the TAR archive may be excluded, and both anchored
and non-anchored exclude files are supported (see section 6 and examples later).
Wildcards (globbing) is supported by default.
Sqfstar can also by invoked by running "mksquashfs - image.sqfs -tar", if
the Sqfstar link isn't available.
1. USAGE EXAMPLES
-----------------
% sqfstar image.sqfs < archive.tar
Create a Squashfs image from archive.tar, using defaults (gzip compression,
128K blocks).
% sqfstar -comp xz -b 1M image.sqfs < archive.tar
As previous, but use XZ compression and 1Mbyte block sizes.
% zcat archive.tgz | sqfstar image.sqfs
Create a Squashfs image from a gzip compressed tar archive.
% sqfstar -root-uid 0 -root-gid 0 image.sqfs < archive.tar
Tar files do not supply a definition for the root directory, and the
default is to make the directory owned/group owned by the user running
Sqfstar. The above command sets the ownership/group ownership to root.
% sqfstar -root-mode 0755 image.sqfs < archive.tar
The default permissions for the root directory is 0777 (rwxrwxrwx). The
above command sets the permissions to 0755 (rwxr-xr-x).
% sqfstar image.sqsh dir1/file1 dir2/file2 < archive.tar
Create a Squashfs image but exclude the files "file1" and "file2".
% sqfstar image.sqsh "... *.[ch]" < archive.tar
Create a Squashfs image but exclude any file matching "*.[ch]" anywhere
in the archive.
2. CHANGING COMPRESSION ALGORITHM AND COMPRESSION SPECIFIC OPTIONS
------------------------------------------------------------------
By default Sqfstar will compress using the GZIP compression algorithm. This
algorithm offers a good trade-off between compression ratio, and memory and time
taken to decompress.
Squashfs also supports LZ4, LZO, XZ and ZSTD compression. LZO offers worse
compression ratio than GZIP, but is faster to decompress. XZ offers better
compression ratio than GZIP, but at the expense of greater memory and time
to decompress (and significantly more time to compress). LZ4 is similar
to LZO. ZSTD has been developed by Facebook, and aims to compress well and
be fast to decompress. You should experiment with the compressors to
see which is best for you.
If you're not building the squashfs-tools and kernel from source, then
the tools and kernel may or may not have been built with support for LZ4, LZO,
XZ or ZSTD compression. The compression algorithms supported by the build of
Sqfstar can be found by typing sqfstar -help.
The full list of compressors available and their compression specific options
are:
Compressors available and compressor specific options:
gzip (default)
-Xcompression-level <compression-level>
<compression-level> should be 1 .. 9 (default 9)
-Xwindow-size <window-size>
<window-size> should be 8 .. 15 (default 15)
-Xstrategy strategy1,strategy2,...,strategyN
Compress using strategy1,strategy2,...,strategyN in turn
and choose the best compression.
Available strategies: default, filtered, huffman_only,
run_length_encoded and fixed
lzo
-Xalgorithm <algorithm>
Where <algorithm> is one of:
lzo1x_1
lzo1x_1_11
lzo1x_1_12
lzo1x_1_15
lzo1x_999 (default)
-Xcompression-level <compression-level>
<compression-level> should be 1 .. 9 (default 8)
Only applies to lzo1x_999 algorithm
lz4
-Xhc
Compress using LZ4 High Compression
xz
-Xbcj filter1,filter2,...,filterN
Compress using filter1,filter2,...,filterN in turn
(in addition to no filter), and choose the best compression.
Available filters: x86, arm, armthumb, powerpc, sparc, ia64
-Xdict-size <dict-size>
Use <dict-size> as the XZ dictionary size. The dictionary size
can be specified as a percentage of the block size, or as an
absolute value. The dictionary size must be less than or equal
to the block size and 8192 bytes or larger. It must also be
storable in the xz header as either 2^n or as 2^n+2^(n+1).
Example dict-sizes are 75%, 50%, 37.5%, 25%, or 32K, 16K, 8K
etc.
zstd
-Xcompression-level <compression-level>
<compression-level> should be 1 .. 22 (default 15)
If the compressor offers compression specific options (all the compressors now
have compression specific options except the deprecated lzma1 compressor)
then these options are also displayed (.i.e. in the above XZ is shown with two
compression specific options). The compression specific options are, obviously,
specific to the compressor in question, and the compressor documentation and
web sites should be consulted to understand their behaviour. In general
the Sqfstar compression defaults for each compressor are optimised to
give the best performance for each compressor, where what constitutes
best depends on the compressor. For GZIP/XZ best means highest compression,
for LZO/LZ4 best means a tradeoff between compression and (de)-compression
overhead (LZO/LZ4 by definition are intended for weaker processors).
3. REDUCING CPU AND I/O USAGE
-----------------------------
By default Sqfstar will use all the CPUs available to compress and create the
filesystem, and will read from the TAR archive and write to the output
filesystem as fast as possible. This maximises both CPU usage and I/O.
Sometimes you don't want Sqfstar to use all CPU and I/O bandwidth. For those
cases Sqfstar supports two complementary options, -processors and -throttle.
The -processors option can be used to reduce the number of parallel compression
threads used by Sqfstar. Reducing this to 1 will create the minimum number of
threads, and this will reduce CPU usage, and that in turn will reduce I/O
(because CPUs are normally the bottleneck).
The -throttle option reduces the speed Sqfstar reads from the TAR archive.
The value is a percentage (obviously from 1 - 100), and 50 will reduce the
read rate by half (the read thread will spend half its time idling), and 75
by three quarters. Reducing the speed of I/O will also reduce the CPU
usage as there is insufficient data rate to use all cores.
Which option should you use? Both will effectively reduce CPU and I/O in
normal cases where intensive use is being made of both I/O and CPUs. But
in edge cases there can be an imbalance where reducing one has no effect, or
it can't be reduced any further. For instance when there is only 1 or 2 cores
available, setting -processors to the minimum of 1 may still use too much
CPU. Additionally if your input source is slow Sqfstar may still max it out
with -processors set to the minimum of 1. In this case you can use -throttle
in addition to -processors or on its own.
4. CHANGING GLOBAL COMPRESSION DEFAULTS USED IN SQFSTAR
-------------------------------------------------------
There are a large number of options that can be used to control the compression
in Sqfstar. By and large the defaults are the most optimum settings and should
rarely need to be changed.
Note, this does not apply to the block size, increasing the block size from
the default of 128 Kbytes will increase compression (especially for the XZ and
ZSTD compressors) and should increase I/O performance too. However, a block
size of greater than 128 Kbytes may increase latency in certain cases (where the
filesystem contains lots of fragments, and no locality of reference is
observed). For this reason the block size default is configured to the less
optimal 128 Kbytes. Users should experiment with 256 Kbyte sizes or above.
The -b option allows the block size to be selected, both "K" and "M" postfixes
are supported, this can be either 4K, 8K, 16K, 32K, 64K, 128K, 256K, 512K or
1M bytes.
The -noI, -noD, -noF and -noX options can be used to force Sqfstar to not
compress inodes/directories, data, fragments and extended attributes
respectively. Giving all options generates an uncompressed filesystem.
The -no-fragments option tells Sqfstar to not generate fragment blocks. A
fragment block consists of multiple small files (all smaller than the block
size) packed and compressed together. This produces much better compression
than storing and compressing these files separately. It also typically
improves I/O as multiple files in the same directory are read at the same time.
You don't want to enable this option unless you fully understand the effects.
The -no-tailends option tells Sqfstar to not pack file tailends into fragment
blocks. Normally a file will not be a multiple of the block size, and so
there were always be a tail which doesn't fit fully into a data block. This
tailend is by default packed into fragment blocks. Enabling this option will
reduce compression.
The -no-duplicates option tells Sqfstar to not check the files being added to
the filesystem for duplicates. This can result in quicker filesystem generation
although obviously compression will suffer badly if there is a lot of duplicate
files.
5. SPECIFYING THE UIDs/GIDs USED IN THE FILESYSTEM
--------------------------------------------------
By default files in the generated filesystem use the ownership of the file
stored in the TAR archive. TAR archives depending on the TAR format stores
ownership in two ways. The early V7 format only stored a numeric UID and GID.
If Sqfstar is reading a V7 archive these are used. Later ustar and PAX
archives can also store a string user name and group name. If these are
present and are recognised by the system (i.e. can be mapped to a UID and GID),
their UID and GID is used. If a user name or group name is not recognised
by the system, the numeric UID or GID is used.
The -all-root option forces all file UIDs/GIDs in the generated Squashfs
filesystem to be root. This allows root owned filesystems to be built without
root access on the host machine.
The "-force-uid value" option forces all files in the generated Squashfs
filesystem to be owned by value, where value can either be a user name or a
numeric UID.
The "-force-gid value" option forces all files in the generated Squashfs
filesystem to be group owned by value, where value can be either a group name or
a numeric GID.
6. EXCLUDING FILES FROM THE FILESYSTEM
--------------------------------------
Sqfstar can exclude files from the TAR archive, so that they don't appear in
the Squashfs filesystem. Exclude files can be directly specified on
the command line (immediately after the FILESYSTEM argument), or the -ef
option can be used to specify an exclude file, with one exclude file per line.
Exclude files by default use wildcard matching (globbing) and can match on
more than one file (if wildcards are used). Regular expression matching
can be used instead by specifying the -regex option. In most cases wildcards
should be used rather than regular expressions because wildcard matching
behaviour is significantly easier to understand!
In addition to wildcards/regex expressions, exclude files can be "anchored" or
"non-anchored". An anchored exclude is one which matches from the root of the
directory and nowhere else, a non-anchored exclude matches anywhere. For
example given the directory hierarchy "a/b/c/a/b", the anchored exclude "a/b"
will match "a/b" at the root of the directory hierarchy, but it will not match
the "/a/b" sub-directory within directory "c", whereas a non-anchored exclude
would.
A couple of examples should make this clearer.
Anchored excludes
1. sqfstar image.sqsh 'test/*.gz' < archive.tar
Exclude all files matching "*.gz" in the top level directory "test".
2. sqfstar image.sqsh '*/[Tt]est/example*' < archive.tar
Exclude all files beginning with "example" inside directories called
"Test" or "test", that occur inside any top level directory.
Using extended wildcards, negative matching is also possible.
3. sqfstar image.sqsh 'test/!(*data*).gz' < archive.tar
Exclude all files matching "*.gz" in top level directory "test",
except those with "data" in the name.
Non-anchored excludes
By default excludes match from the top level directory, but it is
often useful to exclude a file matching anywhere in the source directories.
For this non-anchored excludes can be used, specified by pre-fixing the
exclude with "...".
Examples:
1. sqfstar image.sqsh '... *.gz' < archive.tar
Exclude files matching "*.gz" anywhere in the source directories.
For example this will match "example.gz", "test/example.gz", and
"test/test/example.gz".
2. sqfstar image.sqsh '... [Tt]est/*.gz' < archive.tar
Exclude files matching "*.gz" inside directories called "Test" or
"test" that occur anywhere in the source directories.
Again, using extended wildcards, negative matching is also possible.
3. sqfstar image.sqsh '... !(*data*).gz' < archive.tar
Exclude all files matching "*.gz" anywhere in the source directories,
except those with "data" in the name.
7. FILTERING AND ADDING EXTENDED ATTRIBUTES (XATTRs)
----------------------------------------------------
Sqfstar has a number of options which allow extended attributes (xattrs) to be
filtered from the TAR archive or added to the created Squashfs filesystem.
The -no-xattrs option removes any extended attributes which may exist in the
TAR archive, and creates a filesystem without any extended attributes.
The -xattrs-exclude option specifies a regular expression (regex), which
removes any extended attribute that matches the regular expression from all
files. For example the regex '^user.' will remove all User extended attributes.
The -xattrs-include option instead specifies a regular expression (regex)
which includes any extended attribute that matches, and removes anything
that does't match. For example the regex '^user.' will only keep User
extended attributes and anything else will be removed.
Sqfstar also allows you to add extended attributes to files in the Squashfs
filesystem using the -xattrs-add option. This option takes an xattr name and
value pair separated by the '=' character.
The extended attribute name can be any valid name and can be in the namespaces
security, system, trusted, or user. User extended attributes are added to files
and directories (see man 7 xattr for explanation), and the others are added to
all files.
The extended attribute value by default will be treated as binary (i.e. an
uninterpreted byte sequence), but it can be prefixed with 0s, where it will be
treated as base64 encoded, or prefixed with 0x, where it will be treated as
hexidecimal.
Obviously using base64 or hexidecimal allows values to be used that cannot be
entered on the command line such as non-printable characters etc. But it
renders the string non-human readable. To keep readability and to allow
non-printable characters to be entered, the 0t prefix is supported. This
encoding is similar to binary encoding, except backslashes are specially
treated, and a backslash followed by three octal digits can be used to encode
any ASCII character, which obviously can be used to encode non-printable values.
The following four command lines are equivalent
sqfstar -xattrs-add "user.comment=hello world" image.sqfs
sqfstar -xattrs-add "user.comment=0saGVsbG8gd29ybGQ=" image.sqfs
sqfstar -xattrs-add "user.comment=0x68656c6c6f20776f726c64"
sqfstar -xattrs-add "user.comment=0thello world" image.sqfs
Obviously in the above example there are no non-printable characters and so
the 0t prefixed string is identical to the first line. The following three
command lines are identical, but where the space has been replaced by the
non-printable NUL '\0' (null character).
sqfstar -xattrs-add "user.comment=0thello\000world" image.sqfs
sqfstar -xattrs-add "user.comment=0saGVsbG8Ad29ybGQ=" image.sqfs
sqfstar -xattrs-add "user.comment=0x68656c6c6f00776f726c64" image.sqsh
8. PSEUDO FILE SUPPORT
----------------------
Sqfstar supports pseudo files, these allow files, directories, character
devices, block devices, fifos, symbolic links, hard links and extended
attributes to be specified and added to the Squashfs filesystem being built,
rather than requiring them to be present in the TAR archive. This, for example,
allows device nodes to be added to the filesystem without requiring root access.
Pseudo files also support "dynamic pseudo files" and a modify operation.
Dynamic pseudo files allow files to be dynamically created when Sqfstar
is run, their contents being the result of running a command or piece of
shell script. The modifiy operation allows the mode/uid/gid of an existing
file in the source filesystem to be modified.
Two Sqfstar options are supported, -p allows one pseudo file to be specified
on the command line, and -pf allows a pseudo file to be specified containing a
list of pseduo definitions, one per line.
8.1 CREATING A DYNAMIC FILE
---------------------------
Pseudo definition
Filename f mode uid gid command
mode is the octal mode specifier, similar to that expected by chmod.
uid and gid can be either specified as a decimal number, or by name.
command can be an executable or a piece of shell script, and it is executed
by running "/bin/sh -c command". The stdout becomes the contents of
"Filename".
Examples:
Running a basic command
-----------------------
/somedir/dmesg f 444 root root dmesg
creates a file "/somedir/dmesg" containing the output from dmesg.
Executing shell script
----------------------
RELEASE f 444 root root \
if [ ! -e /tmp/ver ]; then \
echo 0 > /tmp/ver; \
fi; \
ver=`cat /tmp/ver`; \
ver=$((ver +1)); \
echo $ver > /tmp/ver; \
echo -n `cat /tmp/release`; \
echo "-dev #"$ver `date` "Build host" `hostname`
Creates a file RELEASE containing the release name, date, build host, and
an incrementing version number. The incrementing version is a side-effect
of executing the shell script, and ensures every time Sqfstar is run a
new version number is used without requiring any other shell scripting.
The above example also shows that commands can be split across multiple lines
using "\". Obviously as the script will be presented to the shell as a single
line, a semicolon is need to separate individual shell commands within the
shell script.
Reading from a device (or fifo/named socket)
--------------------------------------------
input f 444 root root dd if=/dev/sda1 bs=1024 count=10
Copies 10K from the device /dev/sda1 into the file input. Ordinarily Sqfstar
given a device, fifo, or named socket will place that special file within the
Squashfs filesystem, the above allows input from these special files to be
captured and placed in the Squashfs filesystem.
8.2 CREATING A BLOCK OR CHARACTER DEVICE
----------------------------------------
Pseudo definition
Filename type mode uid gid major minor
Where type is either
b - for block devices, and
c - for character devices
mode is the octal mode specifier, similar to that expected by chmod.
uid and gid can be either specified as a decimal number, or by name.
For example:
/dev/chr_dev c 666 root root 100 1
/dev/blk_dev b 666 0 0 200 200
creates a character device "/dev/chr_dev" with major:minor 100:1 and
a block device "/dev/blk_dev" with major:minor 200:200, both with root
uid/gid and a mode of rw-rw-rw.
8.3 CREATING A DIRECTORY
------------------------
Pseudo definition
Filename d mode uid gid
mode is the octal mode specifier, similar to that expected by chmod.
uid and gid can be either specified as a decimal number, or by name.
For example:
/pseudo_dir d 666 root root
creates a directory "/pseudo_dir" with root uid/gid and mode of rw-rw-rw.
8.4 CREATING A SYMBOLIC LINK
----------------------------
Pseudo definition
Filename s mode uid gid symlink
uid and gid can be either specified as a decimal number, or by name.
Note mode is ignored, as symlinks always have "rwxrwxrwx" permissions.
For example:
symlink s 0 root root example
Creates a symlink "symlink" to file "example" with root uid/gid.
8.5 CREATING HARD LINKS (FILE REFERENCES)
-----------------------------------------
The "f" Pseudo definition allows a regular file to be created from the output of
a command (or shell). Often this is used to reference a file outside the source
directories by executing "cat", e.g.
README f 0555 0 0 cat /home/phillip/latest-version/README
Because this is a quite frequent use of the definition, an alternative faster
"File reference" or Hard Link Pseudo definition exists:
README l /home/phillip/latest-version/README
Will create a reference to "/home/phillip/latest-version/README",
and obviously the timestamp/mode and owership will be used.
The definition also can be used to create additional references to files
within the source directories. For instance if "phillip/latest/README" was a
file being added to the filesystem, then
README l phillip/latest/README
Will create a Hard Link (and increment the nlink count on the inode).
In both cases, the path to the file being referenced is the system
filesystem path, and can be absolute (prefixed with /), or relative
to the current working directory.
There is an additional 'L' Pseudo definition, which closes a loophole in
the above 'l' definition. The 'l' Pseudo definition cannot create references
or Hard Links to files created by Pseudo definitions, because by
definition they do not exist in the system filesystem.
with 'L' the referenced file is expected to be a Pseudo file, and in this case
the path is taken to be from the root of the Squashfs filesystem being created,
e.g.
char-dev c 0555 0 0 1 2
link L char-dev
Will create a Hard Link named "link" to the character device called "char-dev"
created by the previous Pseudo definition.
8.6 CREATING SOCKETS/FIFOS
--------------------------
Pseudo definition
filename i mode uid gid [s|f]
To create a Unix domain socket, 's' should be used, i.e.
filename i 0777 root root s
and to create a FIFO, 'f' should be used, i.e.
filename i 0777 root root f
8.7 ADDING EXTENDED ATTRIBUTES TO FILES
---------------------------------------
Pseudo definition
filename x name=val
Will add the extended attribute <name> to <filename> with <val > contents. See
Section 7 for a description of the <val> formats supported.
8.8 MODIFYING ATTRIBUTES OF AN EXISTING FILE
--------------------------------------------
Pseudo definition
Filename m mode uid gid
mode is the octal mode specifier, similar to that expected by chmod.
uid and gid can be either specified as a decimal number, or by name.
For example:
dmesg m 666 root root
Changes the attributes of the file "dmesg" in the filesystem to have
root uid/gid and a mode of rw-rw-rw, overriding the attributes obtained
from the TAR archive.
9. EXTENDED PSEUDO FILE DEFINITIONS WITH TIMESTAMPS
---------------------------------------------------
The Pseudo file definitions described above do not allow the timestamp
of the created file to be specified, and the files will be timestamped
with the current time.
Extended versions of the Pseudo file definitions are supported which
take a <time> timestamp. These are distinquished from the previous
definitions by using an upper-case type character. For example the "D"
definition is identical to the "d" definition, but it takes a <time>
timestamp.
The list of extended definitions are:
filename F time mode uid gid command
filename D time mode uid gid
filename B time mode uid gid major minor
filename C time mode uid gid major minor
filename S time mode uid gid symlink
filename I time mode uid gid [s|f]
filename M time mode uid gid
<time> can be either an unsigned decimal integer (which represents the
seconds since the epoch of 1970-01-01 00:00 UTC), or a "date string"
which is parsed and converted into an integer since the epoch, by calling
the "date" command.
Because most date strings have spaces, they will need to be quoted, and if
entered on the command line, these quotes will need to be protected from the
shell by backslashes, i.e.
% sqfstar img.sqfs -p "file D \"1 jan 1980\" 0777 phillip phillip" < archive.tar
Obviously anything "date" accepts as a valid string can be used, such as
"yesterday", "last week" etc.
10. MISCELLANEOUS OPTIONS
------------------------
The -info option displays the files/directories as they are compressed and
added to the filesystem. The original uncompressed size of each file
is printed, along with DUPLICATE if the file is a duplicate of a
file in the filesystem.
The -nopad option informs Sqfstar to not pad the filesystem to a 4K multiple.
This is performed by default to enable the output filesystem file to be mounted
by loopback, which requires files to be a 4K multiple. If the filesystem is
being written to a block device, or is to be stored in a bootimage, the extra
pad bytes are not needed.