Migrating to a new Liquidsoap version¶
In this page, we list the most common catches when migrating to a new version of Liquidsoap.
Generalities¶
If you are installing via opam
, it can be useful to create a new switch to install
the new version of liquidsoap
. This will allow to test the new version while keeping
the old version around in case you to revert to it.
More generally, we recommend to always keep a version of your script around and also to make sure that you test your new script with a staging environment that is close to production. Streaming issues can build up over time. We do our best to release the most stable possible code but problems can arise from many reasons so, always best to first to a trial run before putting things to production!
From 2.1.x to 2.2.x¶
References¶
The !x
notation for getting the value of a reference is now deprecated. You
should write x()
instead. And x := v
is now an alias for x.set(v)
(both
can be used interchangeably).
Icecast and Shoutcast outputs¶
output.icecast
and output.shoutcast
are some of our oldest operators and were in dire need of some
cleanup so we did it!
We applied the following changes:
- You should now use
output.icecast
only for sending to icecast servers andoutput.shoutcast
only for sending to shoutcast servers. All shared options have been moved to their respective specialized operator. - Old
icy_metadata
argument was renamed tosend_icy_metadata
and changed to a nullablebool
.null
means guess. - New
icy_metadata
argument now returns a list of metadata to send with ICY updates. - Added a
icy_song
argument to generate default"song"
metadata for ICY updates. Defaults to<artist> - <title>
when available, otherwiseartist
ortitle
if available, otherwisenull
, meaning don’t add the metadata. - Cleaned up and removed parameters that were irrelevant to each operator, i.e.
icy_id
inoutput.icecast
and etc. - Made
mount
mandatory andname
nullable. Usemount
asname
whenname
isnull
.
Harbor HTTP server and SSL support¶
The API for registering HTTP server endpoint and using SSL was completely rewritten. It should be more flexible and provide node/express like API for registering endpoints and middleware. You can checkout the harbor HTTP documentation for more details. The Https support section also explains the new SSL/TLS API.
Timeout¶
We used to have timeout values labelled timeout
or timeout_ms
, some of these would be integer and
in milliseconds, other floating point and in seconds etc. This was pretty confusing so, now all timeout
settings and arguments have been unified to be named timeout
and hold a floating point value representing
a number of seconds.
In most cases, your script will not execute until you have updated your custom timeout
values but you should also review all of them to make sure that they follow the new
convention.
reopen_*
arguments in output.file
and similar¶
The reopen_*
arguments have been unified in output.file
, output.pipe
and similar operators. Instead
of 3 different arguments for metadata, errors and regular reloads, which was making the logic pretty hard to
understand, now a single should_reload
callback is given.
The callback receives two optional arguments, metadata
and error
and is called in 3 different conditions:
- When a new metadata comes up, in which case the
metadata
argument is present - When an error occurs, in which case the
error
argument is present - As the stream is playing unless a reopen is already scheduled, in which case both
metadata
anderror
arguments arenull
Each time that the callback is executed, if it returns a positive float
number, then a reload is scheduled
after that value (in seconds) has passed.
For instance, to reload after 2.
seconds on errors and immediately on metadata
but never otherwise, you can do:
def should_reload(~metadata, ~error) =
if null.defined(error) then
print("Reloading on error: #{error}")
2.
elsif null.defined(metadata) then
print("Reloading on metadata")
0.
else
null()
end
end
Another use-case is to reload on top of each hour and do nothing on metadata
or error
. This can be a little more tricky because the callback
is called on every audio frame so several times per seconds. To prevent multiple reloads in a row, we block
all reloads after the first one until minute 00
has passed:
# Use a ref to reload only once on minute 00:
has_reloaded = ref(false)
def should_reload(~metadata, ~restart) =
if null.defined(metadata) or null.defined(restart) then
null()
elsif 00m and not has_reloaded() then
has_reloaded := true
0.
else
if not 00m then
# Reset has_reloaded
has_reloaded := false
end
null()
end
end
Metadata overrides¶
Some metadata overrides have been made to reset on track boundaries. Previously, those were permanent even though they
were documented as only applying to the current track. If you need to keep the previous behavior, you can used the
persist_overrides
parameters (persis_override
for cross
/crossfade
).
The list of concerned metadata is:
"liq_fade_out"
"liq_fade_skip"
"liq_fade_in"
"liq_cross_duration"
"liq_fade_type"
JSON rendering¶
The confusing let json.stringify
syntax has been removed as it did not provide any feature not already covered by either
the json.stringify()
function or the generic json()
object mapper. Please use either of those now.
Default character encoding in output.{harbor,icecast,shoutcast}
¶
Default encoding for output.harbor
, output.icecast
and output.shoutcast
metadata has been changed to UTF-8
in all cases.
Legacy systems used to expect ISO-8859-1
(also known as latin1
) for metadata inserted into mp3
streams via the icy
mechanism.
It seems that, nowadays, most software expect UTF-8
out of the box, including for legacy systems that previously
assumed other encodings. Therefore, by changing this default value, we try to match exectations of the largest
number of users of our software.
If you are using one of these outputs, make sure to test this assumptions with your listners’ clients. If needed, the characters encoding can be set to a different value using the operator’s parameters.
Decoder names¶
Decoder names have been converted to lowercase. If you were relying on specific settings for decoders priority/ordering, you will need to convert them to lowercase, for instance:
settings.decoder.decoders.set(["FFMPEG"])
becomes:
settings.decoder.decoders.set(["ffmpeg"])
Actually, because of the above change in references, this even becomes:
settings.decoder.decoders := ["ffmpeg"]
strftime
¶
Add file-based operators do not support strftime
type conversions out of the box anymore. Instead, you should use explicit conversions using time.string
. This means that this script:
output.file("/path/to/file%H%M%S.wav", ...)
becomes:
output.file({time.string("/path/to/file%H%M%S.wav")}, ...)
Other breaking changes¶
request.duration
now returns anullable
float,null
being value returned when the request duration could not be computed.getenv
(resp.setenv
) has been renamed toenvironment.get
(resp.environment.set
).
From 2.0.x to 2.1.x¶
Regular expressions¶
First-class regular expression are introduced and are used to replace the following operators:
string.match(pattern=<regexp>, <string>
is replaced by:r/<regexp>/.test(<string>)
string.extract(pattern=<regexp>, <string>)
is replaced by:r/<regexp>/.exec(<string>)
string.replace(pattern=<regexp>, <string>)
is replaced by:r/<regexp>/g.replace(<string>)
string.split(separator=<regexp>, <string>)
is replaced by:r/<regexp>/.split(<string>)
Partial application¶
In order to improve performance, avoid some programming errors and simplify the
code, the support for partial application of functions was removed (from our
experience it was not used much anyway). This means that you should now provide
all required arguments for functions. The behavior corresponding to partial
application can of course still be achieved by explicitly abstracting (with
fun(x) -> ...
) over some arguments.
For instance, suppose that we defined the addition function with two arguments with
def add(x,y) =
x + y
end
and defined the successor function by partially applying it to the first argument
suc = add(1)
We now need to explicitly provide the second argument, and the suc
function
should now be defined as
suc = fun(x) -> add(1, x)
or
def suc(x) =
add(1, x)
end
JSON parsing¶
JSON parsing was greatly improved and is now much more user-friendly. You can check out our detailed presentation here.
Runtime evaluation¶
Runtime evaluation of strings has been re-implemented as a type-safe
eval let
decoration. You can now do:
let eval x = "[1,2,3]"
And, just like with JSON parsing, the recommended use is with a type annotation:
let eval (x: [int]) = "[1,2,3]"
Deprecations and breaking changes¶
- The argument
streams_info
ofoutput.file.hls
is now a record. - Deprecated argument
timeout
ofhttp.*
operators. source.on_metadata
andsource.on_track
now return a source as this was the case in previous versions, and associated handlers are triggered only when the returned source is pulledoutput.youtube.live
renamedoutput.youtube.live.rtmp
, removebitrate
andquality
arguments and added a single encoder argument to allow stream copy and more.list.mem_assoc
is replaced bylist.assoc.mem
timeout
argument inhttp.*
operators is replaced bytimeout_ms
.request.ready
is replaced byrequest.resolved
From 1.4.x to 2.0.0¶
audio_to_stereo
¶
audio_to_stereo
should not be required in most situations anymore. liquidsoap
can handle channels conversions transparently now!
auth
function in input.harbor
¶
The type of the auth
function in input.harbor
has changed. Where before, you would do:
def auth(user, password) =
...
end
You would now do:
def auth(params)
user = params.user
password = params.password
...
end
Type errors with lists of sources¶
Now that sources have their own methods, the actual list of methods attached to each source can vary from one to the next. For instance,
playlist
has a reload
method but input.http
does not. This currently confuses the type checker and leads to errors that look like this:
At script.liq, line xxx, char yyy-zzz:
Error 5: this value has type
_ * source(audio=?A, video=?B, midi=?C)
.{
time : () -> float,
shutdown : () -> unit,
fallible : bool,
skip : () -> unit,
seek : (float) -> float,
is_active : () -> bool,
is_up : () -> bool,
log :
{level : (() -> int?).{set : ((int) -> unit)}
},
self_sync : () -> bool,
duration : () -> float,
elapsed : () -> float,
remaining : () -> float,
on_track : ((([string * string]) -> unit)) -> unit,
on_leave : ((() -> unit)) -> unit,
on_shutdown : ((() -> unit)) -> unit,
on_metadata : ((([string * string]) -> unit)) -> unit,
is_ready : () -> bool,
id : () -> string,
selected : (() -> source(audio=?D, video=?E, midi=?F)?)
}
but it should be a subtype of the type of the value at radio.liq, line 122, char 2-21
_ * _.{reload : _}
In such cases, we recommend to give a little nudge to the typechecker by using the (s:source)
type annotation where a list of source is causing the issue. For instance:
s = fallback([
(s1:source),
(s2:source),
(s3:source)
])
This tells the type checker not to worry about the source methods and just focus on what matters, that they are actually sources.. 🙂
Http input and operators¶
In order to provide as much compatibility as possible with the different HTTP procotols and implementation, we have decided
to delegate HTTP support to external libraries which have large scale support and implementation. This means that,
if you have installed liquidsoap
using opam
:
- You need to install the
ocurl
package to enable all HTTP request operators,http.get
,http.post
,http.put
,http.delete
andhttp.head
- You need to install the
ffmpeg
package (version1.0.0
or above) to enableinput.http
- You do not need to install the
ssl
package anymore to enable theirhttps
counter-part. These operators have been deprecated.
Crossfade¶
The parameters for cross
transitions was changed to take advantage of the new module system. Instead of passing multiple arguments
related to the ending and starting track, those are regrouped into a single record. So, if you had a transition like this:
def transition(
ending_dB_level, starting_dB_level,
ending_metadata, starting_metadata,
ending_source, starting_source) =
...
end
You would now do:
def transition(ending, starting) =
# Now you can use:
# - ending.db_level, ending.metadata, ending.source
# - starting.db_level, starting.metadata, starting.source
...
end
Settings¶
Settings are now exported as records. Where you would before write:
set("decoder.decoders", ["MAD", "FFMPEG"])
You can now write:
settings.decoder.decoders.set(["MAD", "FFMPEG"])
Likewise, to get a setting’s value you can now do:
current_decoders = settings.decoder.decoders()
This provides many good features, in particular type-safety.
For convenience, we have added shorter versions of the most used settings. These are all shortcuts to their respective settings
values:
log.level.set(4)
log.file.set(true)
log.stdout.set(true)
init.daemon.set(true)
audio.samplerate.set(48000)
audio.channels.set(2)
video.frame.width.set(720)
video.frame.height.set(1280)
The register
operator could not be adapted to this new API and had to be removed, however, backward-compatible
set
and get
operators are provided. Make sure to replace them as they should be removed in a future version.
Metadata insertion¶
The function insert_metadata
does not return a pair anymore, but a source with
a method named insert_metadata
. This means that you should change the code
fs = insert_metadata(s)
# The function to insert metadata
f = fst(ms)
# The source with inserted metadata
s = snd(ms)
...
# Using the function
f([("artist", "Bob")])
...
# Using the source
output.pulseaudio(s)
to
s = insert_metadata(s)
...
# Using the function
s.insert_metadata([("artist", "Bob")])
...
# Using the source
output.pulseaudio(s)
Request-based queueing¶
Queueing for request-based sources has been simplified. The default_duration
and length
have been removed in favor of
a simpler implementation. You can now pass a prefetch
parameter which tells the source how many requests should be queued
in advance.
Should you need more advanced queueing strategy, request.dynamic.list
and request.dynamic
now export functions to retrieve
and set their own queue of requests.
JSON import/export¶
json_of
has been renamed json.stringify
and of_json
has been renamed json.parse
.
JSON export has been enhanced with a new generic json object export. Associative lists of type (string, 'a)
are now
exported as lists. See our JSON documentation page for more details.
Convenience functions have been added to convert metadata to and from JSON object format: metadata.json.stringify
and
metadata.json.parse
.
Returned types from output operators¶
Starting with liquidsoap 2.0.0
, output operators return the empty value ()
while they previously returned a source.
This helps enforce the fact that outputs should be end-points of your scripting graphs. However, in some cases, this can cause issues while migrating old scripts, in particular if the returned value of an output was used in the script.
The way to fix this is to apply your operator to the source directly underneath the output. For instance, the following clock assignment:
s = ...
clock.assign_new([output.icecast(..., s)])
Should now be written:
s = ...
clock.assign_new([s], ...)
output.icecast(..., s)
Deprecated operators¶
Some operators have been deprecated. For most of them, we provide a backward-compatible support but it is good practice to update your script. You should see logs in your script when running deprecated operatords. Here’s a list of the most important ones:
playlist.safe
is replaced by:playlist(mksafe(..))
playlist.once
is replaced by:playlist
, settingreload_mode
argument to"never"
andloop
tofalse
rewrite_metadata
should be rewritten usingmetadata.map
fade.inital
andfade.final
are not needed anymoreget_process_output
is replaced by:process.read
get_process_lines
is replaced by:process.read.lines
test_process
is replaced by:process.test
system
is replaced by:process.run
add_timeout
is replaced by:thread.run.recurrent
on_blank
is replaced by:blank.detect
skip_blank
is replaced by:blank.skip
eat_blank
is replaced by:blank.eat
strip_blank
is replaced by:blank.strip
which
is replaced by:file.which
register_flow
: flow is no longer maintainedempty
is replaced by:source.fail
file.unlink
is replaced by:file.remove
string.utf8.escape
is replaced by:string.escape
metadata.map
is replaced by:metadata.map
Windows build¶
The windows binary is statically built and, for this reason, we cannot enable both the %ffmpeg
encoder and any encoder that
uses the same underlying libraries, for instance libmp3lame
for mp3
encoding. The technical reason is that both libraries
import the same C symbols, which makes compilation fail.
The %ffmpeg
encoder provides all the functionalities of the internal encoders that conflict with it along with many more format
we do not support otherwise. For this reason, it was decided to enable the %ffmpeg
encoder and disable all other encoders.
This means that, if you were previously using a different encoder than %ffmpeg
, you will need to adapt your script to
use it. For instance, for mp3 encoding with variable bitrate:
%ffmpeg(format="mp3", %audio(codec="libmp3lame", q=7))