Machnikowski, Maciej maciej.machnikowski@intel.com writes:
-----Original Message----- From: Petr Machata petrm@nvidia.com
Machnikowski, Maciej maciej.machnikowski@intel.com writes:
-----Original Message----- From: Petr Machata petrm@nvidia.com
Machnikowski, Maciej maciej.machnikowski@intel.com writes:
-----Original Message----- From: Ido Schimmel idosch@idosch.org
On Thu, Dec 02, 2021 at 03:17:06PM +0000, Machnikowski, Maciej wrote: > > -----Original Message----- > > From: Ido Schimmel idosch@idosch.org > > > > 4. Why these pins are represented as attributes of a netdev > > and not as attributes of the EEC? That is, why are they > > represented as output pins of the PHY as opposed to input > > pins of the EEC? > > They are 2 separate beings. Recovered clock outputs are > controlled separately from EEC inputs.
Separate how? What does it mean that they are controlled separately? In which sense? That redirection of recovered frequency to pin is controlled via PHY registers whereas priority setting between EEC inputs is controlled via EEC registers? If so, this is an implementation detail of a specific design. It is not of any importance to user space.
They belong to different devices. EEC registers are physically in the DPLL hanging over I2C and recovered clocks are in the PHY/integrated PHY in the MAC. Depending on system architecture you may have control over one piece only
What does ETHTOOL_MSG_RCLK_SET actually configure, physically? Say I have this message:
ETHTOOL_MSG_RCLK_SET dev = eth0
- ETHTOOL_A_RCLK_OUT_PIN_IDX = n
- ETHTOOL_A_RCLK_PIN_FLAGS |= ETHTOOL_RCLK_PIN_FLAGS_ENA
Eventually this lands in ops->set_rclk_out(dev, out_idx, new_state). What does the MAC driver do next?
It goes to the PTY layer, enables the clock recovery from a given physical lane, optionally configure the clock divider and pin output muxes. This will be HW-specific though, but the general concept will look like that.
The reason I am asking is that I suspect that by exposing this functionality through netdev, you assume that the NIC driver will do whatever EEC configuration necessary _anyway_. So why couldn't it just instantiate the EEC object as well?
Not necessarily. The EEC can be supported by totally different driver. I.e there are Renesas DPLL drivers available now in the ptp subsystem. The DPLL can be connected anywhere in the system.
> > 5. What is the problem with the following model? > > > > - The EEC is a separate object with following attributes: > > * State: Invalid / Freerun / Locked / etc > > * Sources: Netdev / external / etc > > * Potentially more > > > > - Notifications are emitted to user space when the state of > > the EEC changes. Drivers will either poll the state from > > the device or get interrupts > > > > - The mapping from netdev to EEC is queried via ethtool > > Yep - that will be part of the EEC (DPLL) subsystem
This model avoids all the problems I pointed out in the current proposal.
That's the go-to model, but first we need control over the source as well :)
Why is that? Can you illustrate a case that breaks with the above model?
If you have 32 port switch chip with 2 recovered clock outputs how will you tell the chip to get the 18th port to pin 0 and from port 20 to pin 1? That's the part those patches addresses. The further side of "which clock should the EEC use" belongs to the DPLL subsystem and I agree with that.
So the claim is that in some cases the owner of the EEC does not know about the netdevices?
If that is the case, how do netdevices know about the EEC, like the netdev-centric model assumes?
Anyway, to answer the question, something like the following would happen:
- Ask EEC to enumerate all input pins it knows about
- Find the one that references swp18
- Ask EEC to forward that input pin to output pin 0
- Repeat for swp20 and output pin 1
The switch driver (or multi-port NIC driver) just instantiates all of netdevices, the EEC object, and pin objects, and therefore can set up arbitrary linking between the three.
This will end up with a model in which pin X of the EEC will link to dozens ports - userspace tool would need to find out the relation between them and EECs somehow.
Indeed. If you have EEC connected to a bunch of ports, the EEC object is related to a bunch of netdevices. The UAPI needs to have tools to dump these objects so that it is possible to discover what is connected where.
This configuration will also not change during the lifetime of the EEC object, so tools can cache it.
It's far more convenient if a given netdev knows where it is connected to and which pin can it drive.
Yeah, it is of course possible to add references from the netdevice to the EEC object directly, so that the tool just needs to ask a netdevice what EEC / RCLK source ID it maps to.
This has mostly nothing to do with the model itself.
I.e. send the netdev swp20 ETHTOOL_MSG_RCLK_GET and get the pin indexes of the EEC and send the future message to find which EEC that is (or even return EEC index in RCLK_GET?).
Since the pin index on its own is useless, it would make sense to return both pieces of information at the same time.
Set the recovered clock on that pin with the ETHTOOL_MSG_RCLK_SET.
Nope.
Then go to the given EEC and configure it to use the pin that was returned before as a frequency source and monitor the EEC state.
Yep.
EEC will invoke a callback to set up the tracking. If something special needs to be done to "set the recovered clock on that pin", the handler of that callback will do it.
Additionally, the EEC device may be instantiated by a totally different driver, in which case the relation between its pins and netdevs may not even be known.
Like an EEC, some PHYs, but the MAC driver does not know about both pieces? Who sets up the connection between the two? The box admin through some cabling? SoC designer?
Also, what does the external EEC actually do with the signal from the PHY? Tune to it and forward to the other PHYs in the complex?