fIcy: an icecast/shoutcast stream grabber suite

.. contents::

General Description

fIcy is an icecast/shoutcast stream grabber suite for use under shell environment. Its goal is to cleanly and automatically rip a stream into user customisable files. It will work with ICY compatible streams, allowing you to either to save the stream to disk or to pipe the output to a media player, or even both. fIcy, among other uses, is ideal for batch/unattended recording of radio programs and stream debugging.

The fIcy package includes:

fIcy itself, a stream separator/multiplexer,
fResync, a fast MPEG-resyncing utility,
fPls, a playlist frontend for fIcy.

Files

These files can be found in the latest release of fIcy

`AUTHORS <AUTHORS>`_:

People that have contributed to fIcy.

`NEWS <NEWS>`_:

Release changes.

`COPYING <http://www.gnu.org/licenses/lgpl.html>`_:

License. Please read carefully.

`README <README>`_:

This file. Contains usage description.

`TODO <TODO>`_:

Known bugs, missing/planned improvements.

`FAQ <FAQ>`_:

Frequently asked questions.

`DOWNLOAD <http://sourceforge.net/project/showfiles.php?group_id=104571&package_id=112466>`_:

Latest sources.

Usage

Synopsis::

fIcy [options] <server [port [path]]|url>

        The main program. Takes directly a stream url and dumps the tracks on
        the specified file/s and standard output, depending on the settings.

fPls [options] <file|url> [fIcy options]

        Playlist manager. Reads a playlist (local or remote) and manages
        fIcy retries/timeouts/errors, forwarding the specified flags.

fResync [options] file

        MPEG resyncing utility. Re-aligns head frame headers on dumped or
        broken files. Usually needed for embedded hardware decoders or
        editing software.

fIcy options

  -d            Do not dump the output to stdout.
                Useful when only ripping.

  -e            Enumerate number of ripped files when song metadata
                changes. Uses -o as prefix.

  -E num        Enumeration starting number. When 0, fIcy will try to
                find the highest unused file number automatically.
                Implies -e and requires -o.

  -h            Help

  -c            Do not clobber files.

  -m            Use song title [metadata] when naming files. Uses -o as
                prefix.

  -n            If the file exists create new one with .n as suffix.

  -p            When dumping to stdout consider writing errors as transient.
                Useful when you pipe the output to a media player and
                want to kill it while not interrupting the rip.

  -o file       Dump the output to file.
                Hint: to dump without a prefix use "./"

  -s suffix     Use sfx as a suffix for new files.
                Hint: the .mp3/.m4a extension is NOT implicit.

  -t            Display song title [metadata] while ripping.

  -r            Remove/don't save partial chunks. This will skip the
                first chunk and remove the last one upon
                termination which are (supposedly) incomplete. To use
                in combination with -m or -e.

  -q file       Append file sequence list to file. This may be used to
                rejoin splitted parts with an external tool. fResync
                will use this file in the future.

  -x regex      Save only files whose title (NOT filename) matches
                against this (or one of these) extended regular
                expressions. Multiple -x can be specified on the
                command line to form OR conditions. Dump unaffected.
                Can be combined with -X.

  -X regex      Do NOT save files whose title matches against this
                extended regular expression. Same semantics as -x.

  -I file       Load include/exclude REs from file. Each line must
                be prefixed with + or - to indicate whether it's a
                positive or negative expression (-xX).

  -f expr       Filter titles through the specified sed expression.
                The raw title is passed to the expression (doesn't
                include any additional prefixes/suffixes). As the
                result will be used internally, some limitations apply.
                Read carefully the Filtering_ section.

  -F file       Filter titles through the specified sed script.
                Same semantics as -f, but the expressions are loaded
                from a file instead. Conflicts with -f.

  -M time       Maximum recording time. See Notes_.

  -i time       Maximum network idle time. Stops recording after the specified
                amount of time is passed without network activity.

  -a file       Read authentication credentials from file (the file must
                contain a line of the form user:password). Note that only the
                Basic HTTP authentication scheme is supported.

  -l num        Redirect follow limit. Defaults to 1. 0 disables
                redirection entirely.

fResync options

  -b            By default fResync maps the entire file into memory
                when operating. However this can create problems on
                loaded systems with large files or when simulating.
                This reverts to a buffered I/O mode. This flag is also
                implicit when simulating.

  -s            Simulate the process. Print on the standard output
                the starting sync offset and stream length, but don't
                modify the source file.

  -v            Verbose.

  -n frames     Require/decode at least n valid consecutive frames to
                validate the sync offset. Defaults to 6.

  -m len        Maximum frame length. Defaults to 1597. fResync uses
                this value to determine the maximal region of the file
                to be checked.

fPls options

  -P path       Specify a different name or full path for the fIcy
                executable (defaults to "fIcy").

  -v            Verbose.

  -R max        Specifies the maximal number of retries to do for
                each stream upon connection/read failure.

  -L max        Specifies the maximal number of loops to do for
                the entire playlist (-1 for infinite).

  -T time       Wait time to pause after each failure.

  -M time       Maximum recording time. See Notes_.

  -a file       Read authentication credentials from file. Same as fIcy's when
                loading a playlist via http. The credentials are automatically
                forwarded to fIcy, but you can override them when needed.

  -l num        Redirect follow limit. Same as fIcy's when loading a
                playlist via http.

  -d file       Run as a daemon, redirecting messages to file.
                fIcy's -d option is enforced. As the process is chdir-ed to
                the root directory you also have to specify absolute paths.

Examples

Use fIcy to display ICY titles while playing::

fPls http://example.com:8080/listen.pls -t | mpg123 -

Rip a station until stopped::

fPls -L-1 http://netradio.invalid/listen.pls -s.mp3 -o./ -cmrd

Connect directly to the stream with server:port and /path::

fIcy -s .mp3 -o ./ -md 123.123.123.123 8080 /path/to/stream

Rip an .mp3 stream while playing, but allows the player to be restarted later by using a named fifo (note that you can re/open "fifo" with any player)::

        $ mkfifo fifo
        $ fIcy -p ... > fifo
        $ mpg123 fifo

Record your favourite program "XYZ" usually on-air between 16:30-17:00::

        at 16:30
        fPls -M 30m http://example.com/listen.pls -o program.mp3 -x XYZ
        ^D

Cleanup a ripped and/or damaged mp3 file::

fResync file.mp3

Companion software

`bfr <http://www.glines.org/software/buffer.html>`_:

Audio-oriented rebuffering tool. Ideal for lousy streams.

`mpgedit <http://www.mpgedit.org/>`_:

Frame-level mp3 cutting tool.

Combining these in the right pipeline order is left as an exercise to the user.

DISCLAIMER

We would like to remind you that saving public streams without paying the rights is ILLEGAL unless explicitly stated.

Notes

The output files produced by fIcy may miss audio framing information and headers since the separation does not consider the audio data. For this reason, your player 'may' (but should not) fail to reproduce the dump or output some initial noise: this is expected. fResync can be used to cleanup MPEG files after processing.

You can also use other tools such as mpgedit for cutting the file in arbitrary positions without diminishing the quality. Assuming that your song spans across three files (use -q to know which ones), that's how to proceed::

cat 1.mp3 2.mp3 3.mp3 > temp.mp3 && xmpgedit temp.mp3

Do not resync the files if you're going to post-process them this way: fResync would remove at least one boundary frame on each file, while other tools could also insert extra empty frames to silence the decoder!

The -M flag supported by both fIcy and fPls accepts a time specification in seconds, `HH:MM` or `N minutes/hours/days`. Time starts just after the connection has been established, but without counting further delays. Also beware that -M specified in fPls means `cumulative recording time` (time accumulates across retries/timeouts), while -M specified in fIcy means `single stream recording time` (recording stops at the first error or when the specified time has elapsed).

For bugs, support, documentation or simply suggestions contact the main developer: wave++ <wavexx@users.sf.net>. Our "Concept Developer": SethX <sethx@users.sf.net> has been downsized to QA and testing (blame him for each bug). The fIcy project is located at <http://ficy.sf.net/>. A general support mailing list is hosted on sourceforge_ (low traffic, mostly beta and release announcements).

.. sourceforge: http://sourceforge.net/mail/?group_id=104571

Filtering

Most online radio stations tend to put banners in the title that will be shown in the player, and eventually result in the filename. To overcome to this (and more), fIcy offers the possibility to rewrite each title through a normal sed script via the "-fF" flags. A real sed coprocess is used along the execution so all of sed's power is available, but some limitations apply:

sed must be GNU sed, or support the -u flag (this limitation may eventually be removed in the future).
Each line of input should result in one output line, and ONE ONLY.
Two consecutive identical titles will result in the second one being ignored (thus NOT splitting the stream). Consider this rule, as removing carefully the banner could result in a better separation.
The resulting title will still apply for -xXI as usual.
Please note that the title is filtered, not the filename (which may still have some characters removed/modified). Use -tv to see what is actually sent to the filter.

Filtering examples

As an example, suppose your titles look like this::

Artist - Title (radiobanner)

You can write a sed expression or script containing::

s/ (radiobanner)$//

to remove the trailing part. This facility can also be used to uniform file names, invert Artist/Title positions and so on. Clever use of the pattern space can also be used to merge albums. sed alone can be used to debug expressions, eg::

echo "test title" | sed -e 'expr'

Refer to the sed(1) manual for a complete list of commands you can use.

Installation

fIcy comes with a very simple Makefile that should work on any system using gcc, or IRIX. GNU make or pmake is required. Documentation is generated from these files using rst2html_.

If you need to use a different compiler (for example on OpenBSD), you can call make as follows::

CXX="eg++" make -e

instead of changing manually the Makefile. Please note that, when using gcc, at least g++ >= 3 is required to compile fIcy.

There's no "make install" target. Instead you should copy the generated executables into the final path, usually doing:

cp fIcy fPls fResync /usr/local/bin

Also note that, for "fPls" to work, fIcy must be already installed (be in "PATH") or a full fIcy path must be specified with -P.

.. _rst2html: http://docutils.sourceforge.net/