It took five years for the industry to migrate from PCI Express 1.0 (introduced in 2002) to PCI Express 2.0 (introduced in 2007). Now, the next wave is here - PCI Express 3.0.
Known commonly as PCIe Gen 1, Gen 2 or Gen 3, reflecting on successive generations, it’s the preferred interconnect for scalable systems, devices and applications such as high-end graphics, fault-tolerant clusters, and storage IO sharing networks. PCIe Gen 3 features 8Gb/s bandwidth, which is double that of Gen 2, while preserving compatibility with software and mechanical interfaces. It also has better provisions for reduced power, signal and data integrity, new transmit and receive equalization methods, and clock data recovery.
Anticipating this new wave of Gen 3 systems and devices, new test tools will be needed to ensure the hardware and software are functional and work together. This article will survey those hardware and software tools for validation, compliance and general testing.
Because PCIe Gen 3 requirements are more stringent than those for Gen 2, these new test tools must manage a new level of complexity and challenges. PCIe Gen 3’s dominant feature is to ensure reliable transmission at 8Gb/s in spite of signal distortion, closed eyes and inter-symbol interference. New test tools must account for encoding similar to 128b/130b, effects of receiver and transmitter equalization, and dynamic negotiation of transmitter equalization at boot-up time. At the same time, PCIe Gen 3 is backward compatible with both Gen 2 and Gen 1.
PCIe Gen 3 test tools can span across five parts of the interconnect technology’s specification: electrical, configuration, link, transactions and platform BIOS. Because PCIe Gen 3 supports backward compatibility with Gen 2 and Gen 1, not all existing PCIe test tools have become obsolete; most tools still can be reused.
In general, the types of test tool used will depend on the applications and which tools are available at the time of the product is being qualified. The PCI-SIG is a good source for designers to discover and evaluate compliance tests tools. Its compliance test equipment library shows equipment that it has approved and recommended to others for use. Using equipment from the list reduces risks, saves evaluation time, and ensures common industry-standard test platforms.
Because PCIe Gen 3 tools employ cutting-edge technologies and require substantial investments, careful selection of the test tools used is critical. Eight suggested evaluation criteria are:
1. Industry leadership: Selecting from the top two or three vendors is not only feasible but also recommended. Industry leaders usually have advantages of track records, name brands and market acceptance.
2. Interoperability: Tools must be backward-compatible with PCIe Gen 2 and Gen 1. As new PCIe Gen 3 systems and devices are available, the tools must interoperate with them as well.
3. Usability: The tools must feature user-friendly interfaces and be easy to deploy.
4. Portability: Tools need to be flexible, modular, allow expansion and can interoperate across different multiple operating systems, BIOS, CPUs, and chipsets.
5. Robustness: Test tools must have a high degree of reliability.
6. Compliant specs and clear upgrade path – Winning tools incorporate new spec changes quickly, without negatively impactingcustomer designs.
7. Feature-richness:Test tools not only must run faster and better, they should be flexible to support multiple applications and allow designers to go deep when monitoring the internals of their designs. Additionally, low-power support, low CPU utilization and large memory buffers are desirable. Other compelling features include light in weight, small dimensions, and industry compliance and certification.
8. Warranty, technical support and training: Insurance protection and ancillary support services are critical.
Hardware test tools
Among the many types of hardware tools that can be used for PCIe testing are high-speed oscilloscopes, frequency counters, timers and spectrum analyzers for PLL loop bandwidth testers, BERT scopes for quickly identifying errors in digital bit streams, digitizing analyzers with high sampling rates for measuring jitter and
timings, protocol test cards (PTC) for link/transactions/BIOS testing, and modular PCIe systems for debug and compliance testing. Some tools combine both BERT and oscilloscope functions and have sophisticated
digital de-emphasis (to reduce inter- symbol interference) and clock-recovery functions.
Currently, Synthesys Research (recently part of Tektronix), LeCroy and Agilent are the leading manufacturers of PCIe 3.0 hardware tools.
Table 1 shows that Tektronix offers high-speed oscilloscopes and logic analyzers. However, only LeCroy and Agilent offer both the protocol analyzer and the exerciser. The PTC card for PCIe 3, though not available yet, would be useful for PCI-SIG compliance testing of root complexes and endpoints. PTC card prototypes are expected be available for testing by mid-2011.
Table 1. Announced PCI Express 3.0 hardware test tools.
Together, these test systems enable fine data capture of transactions and decode protocols, and assist with root-cause debugging. They allow engineers to better assess the quality of signals while simplifying the floods of PCIe traffic. With the appropriate software, they make up a powerful platform for debugging and testing all types of PCIe events and trouble spots.
Figure 1 shows Agilent’s Digital Test Console consisting of a laptop (with installed test software), the U4002A console (with U4301A analyzer test blade inside), the U4305A exerciser card, and mid-bus probe cables.
Fig 1: Agilent’s digital test console system.
LeCroy offers a similar system, consisting of the Summit T3-16 protocol analyzer and the Summit Z-3-16 exerciser. Table 2 below compares the two test systems.
Software test tools
There are many software tools for testing different types of applications, such as graphics, storage, embedded
systems, networking, and communications. However, it’s important for designers to understand the general PCIe tools for system-level test and debug.
Similar to hardware tools, the user’s application drives the software tools.
Currently, the top PCIe 3 software tools are packaged with the hardware, such as LeCroy’s T3-16 and Agilent’s Digital Test Console. There are also standalone tools, such as those from PLX Technology, which can help with analyzing signal integrity and generating PCIe traffic.
Software packaged with hardware
Both LeCroy and Agilent require separate software licenses for the protocol analyzer and the exerciser. The software allows data capture, display and analysis and automated testing. The figures below show how the LeCroy software can be used for PCIe testing.
Figure 2 shows a Trace View snapshot of a decoded Memory Read transaction using LeCroy’s CATC software. It shows detailed time stamps, Request IDs, Completer IDs, tags, Addresses, Status, and Data
sizes. The LTSSM Flow Graph in the bottom right also visually shows transition states from L0 to Recovery.
Fig 2: Trace View Example of CATC PCI Express Traffic (courtesy of LeCroy).
By understanding the areas of focus and trigger/filtering sequence, software can help find errors quickly by isolating important traffic, specific errors or data patterns. Figure 3 below shows how a trigger is set up.
Fig 3. Setting triggers to capture events (courtesy of LeCroy)
Software not only shows a specific status of the hardware, but can be used to see flow relationships. Figure 4 below flow control views of credit behavior between the root complexes and the endpoints.
Fig 4. Flow Control Views of credit updates (courtesy of LeCroy)
Tools such as PLX Technology’s visionPak and performancePak can help with assessing signal integrity and generating traffic at 8Gb/s.
visionPak provides SerDes eye width and height, gradient plot and time estimates, which together allow designers to “see” the quality and gradient levels of high-speeds signals. A plot of the SerDes eye width
and height, for example, can be used as an early indicator of potential link issues. A wide-open eye means a very reliable link, while a comparatively closed eye indicates a weak link. Visually, the user will see estimates of eye widths and heights plotted against the time basis (see figure 5).
Fig 5: Graph showing typical SerDes eye width boundary.
By running compliance patterns and generating plots of received data, the user can select which lane to display, and finely “tune” the parameters (such as post cursor emphasis level, receiver equalization, drive level, and user test patterns).
performancePak consists of packet generator and performance monitor tools. The packet generator tool allows designers to create PCIe Gen 3 traffic patterns (such as Memory Reads or Memory Writes operations), set
TLP destination addresses, payload size and data generation (see Figure 6).
Fig 6: Setting SerDes eye patterns.
The packet generator is an inexpensive way to run system stress tests, without using expensive PCIe exercisers. The user can mix and match different sets of programmable transactions, force errors and check for response, create new batches of programmable traffic for competitive/comparative testing and fully saturate their links (see
Fig 7: Creating a TLP.
More standalone software
Figure 8 shows user-programmable traffic consisting of Memory Reads/Writes, Payload Size and PCI Address. The user can generate multiple Memory Read/Write commands and view them together. Simple or sophisticated test scripts can be constructed, saved and recalled as powerful regression test libraries.
Fig 8: Creating a series of commands with packet generator.
The performance monitor tool (see figure 9) allows designers to track real-time link utilization and helps find any weak links and potential bottlenecks. The user can select ingress and/or egress ports to graph and the utility will display lots of useful statistics such as link use, total bytes, total rate, payload read bytes, payload write bytes,
payload total bytes, payload rate and payload average per TLP.
Fig 9: Reading various performance statistics per port
Putting it all together
As PCIe Gen 3 designs get more complex, using a combination of hardware and/or software tools to address technical issues is appropriate. But getting all the components to work together as whole is near impossible
without good test tools. Until common test guidelines and test tools are available, there will be some level of trial-and-error work to determine which equipment is specification-compliant and which isn’t. Industry forums, plug-fests and invitation-only appointments will be the main venue to test for interoperability and compliance.
Most likely, designers will start with one test system or one device in a specific, known environment, then they gradually connect to and test with other devices. Along the way, devices may not link, bugs are found and corrected, and slowly things will get better. This is a normal progress as seen throughout PCIe’s evolution, and the migration from PCIe Gen 2 to Gen 3 is expected to follow that path as well.
Rounding out the edges
This survey of the most advanced test tools for PCIe Gen 3 is by no means definitive, since new tools are still emerging. Designers will find themselves on a learning curve while migrating from PCIe Gen 2 to Gen 3, while vendors release new software tools, early reference platforms, and modular test tools. Additionally, the test
specifications will change to accommodate new addendums and ECOs, so it’s important for designers to keep with the changes.
At the same time, where industry-standard tools are lacking, proprietary tools will grow and fill the void until they themselves become industry-standard tools. Over time, however, designers can look forward to these tools developing more built-in intelligence, if not becoming fully automated turnkey solutions.
Along the way, leading industry heavyweights such as Intel, AMD, Microsoft, HP, Dell, and IBM will play a large role in accelerating the adoption pace of PCIe Gen 3. The invitation-only forums, plug-fests and
compliance workshops they conduct and participate in will continue to provide valuable feedback on PCIe Gen 3 development tools.
Currently, the PCI-SIG is now evaluating Tektronix, LeCroy and Agilent PCIe Gen 3 test tools for inclusion into the PCI-SIG test tool kits.
The new PCI Express 3.0 Compliance Base Board (CBB) and Compliance Load Board should be available toward the second half of 2011. Microsoft will also upgrade the current Windows Logo Kit 1.5 for its WHQL
certification test suite to incorporate the new PCIe Gen 3 specifications and new test kits available for testing would probably be available in 2012.
All these developments bode well for and indicate exciting times ahead for PCIe Gen 3.
About the author:
Roger Lai, (firstname.lastname@example.org) is an applications consultant with PLX Technology, Sunnyvale, Calif.