Skip to content

Latest commit

 

History

History
293 lines (198 loc) · 10.8 KB

linuxptp-gm.md

File metadata and controls

293 lines (198 loc) · 10.8 KB
Raw

LinuxPTP as a grandmaster

LinuxPTP includes several programs that we need to use:

  • ptp4l is the main PTP daemon, which handles synchronizing the PHCs over the network
  • pmc is a PTP management client, which is used to configure ptp4l, when it is being used as a grandmaster
  • ts2phc handles synchronizing a PHC from timestamps generated by the NIC from a PPS input signal; it can get the time-of-day either from NMEA messages or from the system clock
  • phc2sys handles synchronizing the system clock from a PHC; it can also output PHC timestamps that can be used as input by an NTP server

Start by installing the Linux PTP package. It is called linuxptp on both Fedora and Debian and can be installed in the usual way with dnf or apt.

There are a number of different approaches to setting this up. The first section describes in detail a base configuration. This is the simplest approach that I have works reliably on a variety of distributions at the time of writing (2023-09). Subsequent sections describe alternative approaches as deltas this base approach.

Base configuration

In this setup, we get the approximate time from NTP and make that precise using the PPS input on the NIC. This is using tsphc with its "generic" time-of-day source.

chrony service

We assume that chrony in installed with a configuration similar to its default (without any refclocks and with several NTP servers configured using pool or server directives). On Debian, you may need to install it

sudo apt install chrony

Add this refclock to the chrony configuration:

  • In Fedora, I suggest adding confdir /etc/chrony.d/ to /etc/chrony.conf and then copying the refclock .conf file into /etc/chrony.d.
  • In Debian, you can just copy the refclock .conf file into /etc/chrony/conf.d.

If your chrony.conf specifies other servers using the pool or server directive, make sure you have more than one server, so you don't get into a situation where there is no majority for single time source.

Then restart the chrony service. In Fedora

sudo systemctl restart chronyd

In Debian

sudo systemctl restart chrony

At this point, chronyc sources should show the PTP refclock not being used.

#? PTP                           0   0     0     -     +0ns[   +0ns] +/-    0ns

We also need to enable the chrony-wait service:

sudo systemctl enable chrony-wait.service

This is neeed because the ts2phc services depends on the time-sync.target, which needs the chrony-wait service (or something similar) in order to work.

ts2phc service

Install the service file ts2phc.service into the /etc/systemd/system/ directory.

Now install the ts2phc configuration file ts2phc.conf. This goes into /etc/ on Fedora and into /etc/linuxptp/ on Debian.

You may need to edit this using values you found in Verify PPS section:

  • in [enp1s0] change enp1s0 to the right interface name
  • change ts2phc.pulsewidth, if your pulse width isn't 100000000 nanoseconds (i.e. 0.1s)

On Fedora, create /etc/sysconfig/ts2phc containining:

OPTIONS="-f /etc/ts2phc.conf"

Similarly, on Debian, create /etc/default/tsphc containing:

OPTIONS="-f /etc/linuxptp/ts2phc.conf"

Now start and enable the service

sudo systemctl enable --now ts2phc

You can look at the last 30 seconds of log messages from ts2phc using:

sudo journalctl -u ts2phc -S -30s

This should with a few minutes show offsets reducing to tens of nanoseconds.

Also

sudo phc_ctl enp1s0 cmp

should show a value close to -37s (i.e. close to -37000000000 nanoseconds)

ptp4l service

On Debian, copy the service file ptp4l.service into the /etc/systemd/system/ directory (Debian only provides a template unit for ptp4l).

Then edit the ptp4l.conf: on Fedora, this is in /etc; on Debian, it is in /etc/linuxptp.

Many of the configuration options depend on the PTP profile you are using. Suitable configuration files for many profiles can be found in /usr/share/doc/linuxptp/configs.

To get started, I suggest using the LinuxPTP defaults for most options, but a few things to be changed. You can either edit the existing configuration file or copy this one.

  • We need to specify the interfaces on which LinuxPTP should run: this is done by specifying a section in the form [enp1s0]. An empty section tells LinuxPTP to run on that interface.
  • We need to make sure LinuxPTP runs as a server (grandmaster).
    • The Fedora ptp4l.cong has slaveOnly 1; we need to remove that or change it to 0.
    • We also need to add masterOnly 1; with LinuxPTP 4.0 you can use serverOnly 1 (the point of this is to prevent ptp4l from trying to control the PHC, since that is likely to interfere with ts2phc)

Then enable and start ptp4l:

sudo systemctl enable --now ptp4l.service

Check that it started OK:

sudo systemctl status ptp4l.service

ptp4l-gm service

This service is used to configure ptp4l with settings needs to be make it work properly as a grandmaster.

Copy the service file ptp4l-gm.service into the /etc/systemd/system/ directory.

Then enable and start it.

sudo systemctl enable --now ptp4l-gm.service

Note that it sleeps for 10 seconds when it starts (this is a hack). Check that it worked OK:

sudo journalctl -u ptp4l-gm.service

It should show something like:

Sep 22 11:18:50 halloumi systemd[1]: Starting ptp4l-gm.service - Change the settings of ptp4l to work as a grandmaster.>
Sep 22 11:19:00 halloumi bash[4310]: sending: SET GRANDMASTER_SETTINGS_NP
Sep 22 11:19:00 halloumi bash[4310]:         6805ca.fffe.802b29-0 seq 0 RESPONSE MANAGEMENT GRANDMASTER_SETTINGS_NP
Sep 22 11:19:00 halloumi bash[4310]:                 clockClass              6
Sep 22 11:19:00 halloumi bash[4310]:                 clockAccuracy           0xfe
Sep 22 11:19:00 halloumi bash[4310]:                 offsetScaledLogVariance 0xffff
Sep 22 11:19:00 halloumi bash[4310]:                 currentUtcOffset        37
Sep 22 11:19:00 halloumi bash[4310]:                 leap61                  0
Sep 22 11:19:00 halloumi bash[4310]:                 leap59                  0
Sep 22 11:19:00 halloumi bash[4310]:                 currentUtcOffsetValid   1
Sep 22 11:19:00 halloumi bash[4310]:                 ptpTimescale            1
Sep 22 11:19:00 halloumi bash[4310]:                 timeTraceable           1
Sep 22 11:19:00 halloumi bash[4310]:                 frequencyTraceable      0
Sep 22 11:19:00 halloumi bash[4310]:                 timeSource              0x20
Sep 22 11:19:00 halloumi systemd[1]: ptp4l-gm.service: Deactivated successfully.
Sep 22 11:19:00 halloumi systemd[1]: Finished ptp4l-gm.service - Change the settings of ptp4l to work as a grandmaster.

TODO: instead of sleeping, we can repeatedly use phc_ctl cmp command to wait for the delta between the offset to reduce to enough.

phc2sys service

This service feeds PPS samples from the PHC into a shared memory segment where chrony can read them.

The Debian phc2sys service file isn't suitable, so on Debian copy the phc2sys.service file into the /etc/systemd/system directory.

Now copy the file specifying the phc2sys options into /etc/sysconfig/phc2sys on Fedora and /etc/default/phc2sys on Debian.

Then enable and start the service.

sudo systemctl enable --now phc2sys.service

Now check chrony with

chronyc sources

If everything's working, in a few seconds chrony should start using the PTP source.

Firewall

Fedora has a firewall active by default. We will need to allow PTP through the firewall:

sudo firewall-cmd --add-service ptp
sudo firewall-cmd --add-service ptp --permanent

Test client synchronization

On another machine, run:

sudo ptp4l -i enp1s0 -s -q -m

If this says increasing tx_timestamp_timeout may correct this issue, then add a --tx_timestamp_timeout=10. It should say selected best master clock followed by an identifier with 3 hex numbers separated by dots; this is derived from the MAC address on the server. After a while, it should show consistently small offsets (less than 100).

On Fedora, you will need to allow PTP through the firewall on the client.

Make sure you don't have anything else touching the PHC (like ts2phc).

Use SOCK refclock with phc2sys

This is an alternative approach in which phc2sys uses the SOCK refclock instead of the SHM refclock to provide data to chrony. This is supported in LinuxPTP 4.0.

TODO: explain details

Use nmea time-of-day source in ts2phc

This is an alternative approach in which ts2phc gets the time-of-date by reading NMEA messages from the GPS serial connection. Unfortunately this ts2phc features does not appear not work reliably when used with a NIC, such as the i210, that timestamps both pulse edges, in either version 3 or 4.0.

If this is fixed, then to make use of it:

  • edit ts2phc.service to
    • use -s nmea instead of -s generic
    • comment out the After=time-sync.target line
  • edit the refclock line in the chrony conf to remove the pps option

A further variation here is not to use chrony.

TODO: We don't really want ts2phc.service to depend on network-online. Instead it needs to depend on the devices being available. We can udev rules to add a systemd tag to the relevant devices.

Time of day from gpsd via chrony

This is an alternative approach in which chrony gets time-of-day from the GPS via gpsd.

TODO: explain details

Allow chrony server to use hardware timestamping

This is an alternative approach in which ts2phc and ptp4l use vclocks, so as to enable chrony to make use of hardware timestamping.

Chrony can transfer time over NTP more accurately when it is using hardware timestamping. Hardware timestamping in chrony requires the PHC to be free-running, which means it is not kept in sync with the actual time, whereas PTP requires it to be in sync.

To make them work together on the same PHC, we have to use the vclock feature. A vclock is a virtual PTP hardware clock, which is additional PTP hardware clock device /dev/ptpN on top of a physical PTP hardware clock. PTP will use the vclock, and chrony will use the underlying clock.

To create one vclock on top of /dev/ptp0, we write 1 to /sys/class/ptp/ptp0/n_vclocks. If the newly created PHC is /dev/ptp4, then there will be a directory /sys/class/ptp/ptp0/ptp4. We would then need to use /dev/ptp4 in ts2phc.conf instead of enp1s0, and specify phc_index 4 in ptp4l.conf.

TODO: Figure out how get this to be setup automatically with systemd. Need to wait till the underlying network interface/ptp character device appears, perhaps by using After=sys-subsystem-net-devices-enp1s0.device. This could then produce edited ts2phc.conf and ptp4l.conf in /var/run for use by ptp4l and ts2phc services.