LXI For Collider Signal Monitoring At CERN

The collider operates a pair of counter- rotating particle rings which have crossovers at four experiment sites where particles crash from opposite directions into each other and create the signatures that indicate the presence of short lived particles, and that has recently included the Higgs.

Much of the attention is focussed on the experiments which have to run to capture all the data available in order to identify new particles. However monitoring of the ring itself is also a major undertaking and this is accomplished through the Open Analogue Signal Information System, referred to as OASIS. Signals from the collider monitors can be tapped at many places to make sure all is well in the system.

Even a large budget operation like CERN though has to make sure that its budget matches the finances available from sponsoring governments (and therefore tax payers in many countries), so this system has to be cost effective. The OASIS system uses a set of digitisers to acquire the signals and this is relayed out to users over an Ethernet system, but the digitisers are expensive and there cannot be one digitiser for every monitor signal. A switching system is used to allow OASIS to select which signals to show from the variety of signals available, and that switching system has historically been based on VXI and more recently cPCI solutions, but that is changing.

CERN Upgrade

CERN is undergoing a major 2 year upgrade to its systems so the collider energy can be raised (almost doubled) and more new physics explored. The collider has now been closed down (as of February 2013) for this scheduled upgrade, and more upgrades will occur in the future. Inevitably part of that upgrade process requires the OASIS system to be upgraded.

The monitor signals present some challenges to a switching system. CERN concluded they wanted to be able to select up to 16 out of a maximum of 104 signals available for digitizing at each location. The analogue signals have frequency content to many MHz and there is potential for considerable differences in level from the different monitors. That put major constraints on the allowable crosstalk between channels as well as the bandwidth. If a signal from a high level source was selected and a signal from a low level source at the same time on a different channel then the large signal could breakthrough into the smaller signal and confuse the operators.

Another significant issue for CERN is the sheer size of the collider, you cannot walk from one location to another in any reasonable time – the tunnel is even equipped with bicycles to speed up transportation between locations. Management at a distance is an essential requirement for any solution.

Designing A New OASIS Switch

CERN approached Pickering Interfaces for ideas on a new switching system to be deployed during the scheduled upgrade. The basic requirement was for a matrix with 10’s of MHz of BW and a size of up to 104x16. Discussions made it apparent that crosstalk would be a major concern in any implementation, and the sheer size of the matrix required made it hard to use traditional approaches to solve the problem, meet the performance objectives and meet the budget requirements.

Clearly the cost of the matrix had to be significantly lower than placing a digitiser on each analogue signal. The preferred platform was PCI in an industrial computer but it became very apparent that the fixed modular structure of PCI did not lend itself to this sort of switching system, and the same problems applied to cPCI and PXI.


Figure 1. The CERN requirement requires a matrix to connect up to 104 analogue sources to up to 16 digitisers

To implement a high performance matrix of this type required the switching system to determine the form factor of the final solution - and that ruled out using anything which could be described as fixed modular format. A modular approach was needed to make the matrix system size scalable as different locations required different sizes of matrix – one location might require a 64x16, another might require a 104x16. Systems could also have their requirements changed with time as the number of sensors changed and more (or less) channels added. That strongly indicated that a proprietary scalable modular approach was going to be required, the modules sized to fit the design requirement of the matrix. That encouraged Pickering Interfaces to investigate an LXI route where there is a freedom of size.

LXI Route

LXI had some major advantages for CERN, much of their system was already running Ethernet data connections so using it to manage a matrix was not an issue. LXI control also means that they could access the matrix state over their network without intervening controllers by accessing the LXI products web server.

During discussions another issue arose, the experiments being conducted on the collider are large and expensive operations and the last thing that CERN wanted was to find that a switch in the matrix had developed a fault and was preventing monitoring operations. Knowing that Pickering Interfaces had implemented self-test in both LXI and PXI (called BIRST) CERN requested some sort of self-test in the switching system, and ideally because the switch needed coaxial connectors the test had to be capable of running with the inputs and outputs connected to a non-powered source/load. Being able to initiate and run a self-test remotely would also be a powerful tool for OASIS.

65-110


Figure 2. 65-110 wideband modular chassis 48x16 matrix, with the drawer system out the plugins can be added or removed

The solution arrived at for CERN was the 65-110 Wideband Modular Matrix. The switching matrix is based on a chassis which has a dedicated analogue bus system. Into the chassis a set of plugins can be installed, the left hand pair providing the 16 Y access connections required for the digitisers. A set of X plugins then provide the analogue signal inputs, 8 signals to a plugin module. The number of X plugins can be scaled from just one (8off X connections) up to 13 (104 off X connections), allowing the user to create a matrix of any required size within the chassis constraints. Not installing the second Y plugin allowed Y=8 systems to be created – though CERN had no specific requirements for that configuration other users might find it an advantage if they had smaller system requirements. The design is fully user configurable, plugin modules can be physically installed and uninstalled and the firmware in the LXI controller will recognise the configuration and amend the available matrix size to match the plugin modules installed. The web based soft front panel, a feature strongly encouraged by the LXI standard, allows driverless control of the matrix.


Figure 3. The soft front panel of the 65-110 can be accessed through the LXI configuration pages to either control or monitor the matrix settings. The LXI controller presents the matrix as a single entity, greatly simplifying the user understanding of the setting

The matrix is a modular solution, but the module size is scaled to fit the application rather than to abide by a particular standard. The 65-110 plugin and analogue bus system had to be very carefully designed to maintain the RF performance, and in particular the crosstalk, to ensure it was fit for the application. The RF BW in a typical configuration is above 300MHz, driven largely by the need for low crosstalk, and has excellent VSWR.

Like many modern instruments the modules communicate internally to the LXI controller via a PCIe interface and the LXI controller “virtualises” this as a single matrix, so the LXI controller makes the user task of programming the matrix much easier. The LXI controller hides the complexity of the switch system from the user, the matrix appears as just one entity to the user and not a set of separate sub-assemblies (modules). It behaves like a bench instrument rather than a modular instrument.

The design uses an analogue bus underneath the plugin modules rather than being at the back of the plugin which is normally the case with modular systems – in a matrix it makes much more sense to have the X and Y signals lines at right angles to each other to improve crosstalk and isolation. This is a feature of LXI – there are no particular restraints on the size of the modules or the placement of an analogue bus so Pickering Interfaces were able to design a modular structure to suit the switching requirements.


Figure 4 The web interface on 65-110 allows easy access to the self test facility through the standard LXI configuration pages

The 65-110 includes a self-test facility checks all the signals paths for failed relays (closed, open or high resistance). The design uses low level signals so that the user connections do not need to be disconnected in order to run the test (a time consuming process with over 100 coaxial leads connected, and not very practical given the distances involved) and the self-test can be initiated over the LXI compliant web interface without the use of an external controller program while a user is many kilometres from the matrix. The user simply initiates the test, the embedded LXI controller runs the test and the results can be viewed over the web interface or reported to the user as a file.

A monitor facility is also included in Pickering Interfaces LXI products that allows a user to graphically display the matrix setting without having any program access to the matrix - LXI systems allow the easy creation of systems where multiple controllers are present. One controller can be setting the switch, a different controller can be monitoring what is the settings are without disrupting the programming.

Summary

The CERN requirement shows why LXI provides an excellent platform for the creation of difficult switching systems where the performance objectives are high, the switch is complicated and easy remote access is required. CERN will be making full use of the LXI aspects of 65-110 as part of the OASIS system during its next rounds of experiments running at ever higher collider energies.