Distributing Control with
Ethernet and TCP/IP

It can’t be simple, can it? Industrial networking’s cast of characters is mind boggling. And it’s hard to figure out how they all fit together. Well, consider some of the new industrial Ethernet protocols to see how they facilitate distributed control with intelligent devices. Can’t hurt.

John Rinaldi, Real Time Automation
Perry S. Marshall, Perry S. Marshall & Associates

photo

Industrial Ethernet is a hot topic these days. And although it’s not ideal for sensors, its TCP/IP (Transport Control Protocol/Internet Protocol) peer-to-peer capabilities make it an excellent network for linking controllers and data acquisition systems. Outside the office environment, protocols are necessary for defining the meaning of automation and process data. Without a vendor like Microsoft to dictate these protocols, many within the industry are pushing standards.

This article discusses some of the more popular standards (e.g., EtherNet/IP, Foundation Fieldbus, and Profinet), their relative strengths and weaknesses, and the issues that device manufacturers must consider to support these protocols.

A Fit for Distributed Control

Just as millions of PCs around the world can work independently and then send and receive data over the Internet, distributed control means intelligent control devices work independently but still operate together within a larger system. In process control, engineers have used distributed control systems for decades, with loop controllers operating locally and sending batch information to a mainframe computer. In discrete manufacturing, large plants use hundreds of PLCs linked to a single backbone. Phillip Morris uses serial-to-Ethernet converters to verify that the program in each PLC has not changed. Millions of PLCs use serial networks based on RS-485 to pass data back and forth (see Figure 1).

figure
Figure 1. Distributed control is the sharing of control functions between multiple processors and smart devices, and connecting them with a network. Advanced Ethernet protocols, such as EtherNet/IP, Foundation Fieldbus HSE, and Profinet offer messaging services designed for automation and process control.

But in both of these examples, the data rate is only a few thousand bits per second. Ethernet provides transmission rates of 10, 100, and 1000 Mbps. The faster the data rate, the more transparent the network.

Ethernet is the most common physical layer in distributed control systems, but the transport mechanism itself is secondary. The real deal is the collection of protocols collectively referred to as TCP/IP. This set of protocols accommodates different kinds of data for different kinds of applications. For example, the Simple Mail Transfer Protocol (SMTP) supports mail, the Point-to-Point Protocol (PPP) supports point-to-point applications, and the Hypertext Transfer Protocol (HTTP) supports the Web. Of all the protocols in the TCP/IP suite, TCP and IP are most important to industrial automation.

Switches Segment the Network

Ethernet hubs indiscriminately distribute packets in all directions, but switches pass packets only to their intended destinations. This means that traffic between devices that are close together does not clog up other parts of the network. So a data acquisition system that uses Ethernet I/O can easily coexist with a device passing high-speed motion commands on a different segment.

Intelligent switches make it possible for you to create deterministic segments on your network. Even though Ethernet is by definition not deterministic, thoughtful planning of your network can create zones of determinism. Using intelligent switches to isolate devices on a network segment can provide enough determinism for all but the most demanding applications.

TCP/IP Messaging Services

There are two major general messaging services in TCP/IP: TCP and the User Datagram Protocol (UDP). Using TCP is like sending a certified letter, which provides proof of receipt. The network assumes that the message has been properly received only after the acknowledgement comes back. This virtually guarantees data integrity, but it requires no fewer than seven handshakes to open a connection, send a packet, verify its safe arrival, and close the connection. Of course, if there’s an error, the process has to start over again.

TCP is inherently peer to peer; each node is assumed to be equal to others. It is full duplex, meaning that it can handle simultaneous data streams in both directions.

UDP is like ordinary first-class mail. The message can be sent in a single packet, but there is no acknowledgment of receipt. To continue the mail analogy, a UDP envelope is also less bulky–the UDP protocol has less overhead.

UDP lends itself to master/slave relationships, which are common in I/O networks (e.g., DeviceNet and Profibus). UDP is generally used for real-time applications, such as online streaming and gaming, where missing packets need not be re-sent and would probably be too old if they were. It’s also used where upper-layer protocols handle flow control and data stream checking and correcting.

Automation’s Special Needs

Automation systems perform specific tasks, and data must be represented in a standardized format. Digital and analog values, commands, and recipes can all be represented in a variety of ways, and to avoid a Tower of Babel, definite protocols must be used.

It’s not enough to send packets of data back and forth, because parsing the packets is too tedious. A more elegant way is to use object-oriented data.

There are several major contenders that use objects: Profinet, Foundation Fieldbus High Speed Ethernet (HSE), and EtherNet/IP. Each approaches the problem from a different vantage point. Profinet is not as far along in the commercialization process as Foundation Fieldbus HSE or Ethernet/IP, and some aspects of the specification are still under development. Except for specific process control applications where Foundation Fieldbus is the bus du jour, Ethernet/IP is the most commercially viable solution on the market. It has a small but growing list of supporting vendors.

Profinet

figure

Figure 2. Profinet is an object-oriented architecture designed to allow the distribution of programs, device profiles, and data across a large networked system, similar to the way Microsoft Office applications, programs, and objects can be shared across an office LAN.

This protocol is the Profibus Trade Organization’s answer to the need for interoperability between automation devices and subsystems linked together via Ethernet (see Figure 2). To understand what Profinet is, you must understand what it is not. Profinet is not the Profibus protocol on Ethernet in the same way that Modbus/TCP is the old familiar Modbus on Ethernet. And Profinet is not really a fieldbus as the term is normally understood.

An analogy to the office environment may help you understand what Profinet is intended to do. You work in an office with a PC at your desk, networked with a dozen other PCs and a file server. Your office LAN (Ethernet 100Base–T) is linked to a T1 Internet line. You open Microsoft Word and create a complex document. You write some text and create some tables. Your coworker Jeff has a PowerPoint presentation on his PC; you open it via the network and copy and paste two different graphs into your Word document, which are transferred intact as objects. Your other coworker Leslie has an Excel spreadsheet, which you also open remotely and embed in your document–it’s as simple as cut and paste. In this case, the Excel data are not static, they’re live: Leslie updates the spreadsheet every Tuesday, and every time you open your document, it’s going to retrieve the latest data from the document on her PC. You get on the Internet and copy and paste text and graphics from a Web site, and your document also contains hyperlinks to other Web sites.

Behind this transparency among applications is a complex object model created by Microsoft. Savvy PC users are quite accustomed to this level of sophistication and its benefits. This expectation naturally extends to the integration of business applications throughout an entire company and to devices in an automation system. Profinet in fact uses Microsoft COM (Communication Object Model) and DCOM (Distributed Communication Object Model) to extend this transparency to all devices on a TCP/IP network, further defining object models for many kinds of device and programming parameters.

And rather than being specific to only one manufacturer’s hardware or software (as is often the case with Microsoft), Profinet purports to be an industry standard available to all Profibus members. Profinet is an open communications, multivendor engineering model. This means that a preconfigured, preprogrammed, and pretested machine, such as a transport conveyor, can be set up using vendor-specific electrical devices and applications as it has been in the past.

Foundation Fieldbus HSE

This protocol uses the Foundation Fieldbus H1 process control protocol on TCP/IP. Foundation Fieldbus H1 is a sophisticated, object–oriented protocol that operates at 31.25 Kbps on standard 4–20 mA wiring. It uses multiple messaging formats and allows a controller to recognize a rich set of configuration and parameter information (a device description) from devices plugged into the bus. Foundation Fieldbus even allows a device to transmit parameters relating to the estimated reliability of a particular piece of data. The protocol uses a scheduler to guarantee the delivery of messages, so issues of determinism and repeatability are solidly addressed. Each segment of the network contains one scheduler.

Foundation Fieldbus HSE is the same H1 protocol, but instead of 31.25 Kbps, it runs on TCP/IP at 100 Mbps. HSE provides the same services and transparency of network objects but operates at a higher level.

Foundation Fieldbus is tailored for the process-control industry and will likely be the dominant Ethernet I/O standard in that arena. Installations in this segment of the world typically have the following characteristics:

  • Large campuses (e.g., chemical refineries), with large numbers of nodes
  • Large packets data that don’t have to move quickly
  • Large quantities of analog data
  • Explosionproof requirement (Class I, Div. II)

Foundation Fieldbus H1 links local islands of transducers and actuators; HSE links controllers and transmits a high level of information over large distances.

Ethernet/IP

Ethernet/IP is based on a standard used in DeviceNet and ControlNet called the Control and Information Protocol (CIP). This standard organizes networked devices as a collection of objects and defines access, object behavior, and extensions, which allows widely disparate devices to be accessed using a common mechanism.

CIP is carefully mapped to both TCP/IP and UDP. It incorporates the following message hierarchies and scheduling mechanisms:

  • Exchange of basic I/O, PLC style. All control networks have this capability: exchanging data with racks of I/O, for example, or collecting readings from a temperature controller or encoder. This is handled in a simple client/server relationship using UDP.
  • Upload/download of parameters and set points; transfer of programs and recipes. In DeviceNet, these are called explicit messages and are sent only sporadically. An Ethernet/IP temperature controller maps its data in a similar way. Ethernet/IP uses TCP/IP for this task.
  • Polled, cyclic, and event-driven data.
  • One-to-one, one-to-many, and broadcast. TCP/IP is inherently one-to-one (and peer-to-peer), but this poses a significant disadvantage when many devices must be updated in a short period of time because each connection eats up valuable milliseconds. UDP allows messages to be sent to many nodes simultaneously, and this saves time.

More than 300 vendors support CIP in their products. Using the technology in Ethernet/IP means that it is based on a widely understood, broadly implemented, and certifiable standard that does not require a new-technology shakedown period. Four independent groups have joined forces to develop and promote Ethernet/IP as a public-domain Ethernet application layer for industrial automation. These groups include the Open DeviceNet Vendor Association, the Industrial Automation Open Networking Association, ControlNet International, and the Industrial Ethernet Association. Ethernet/IP is considerably more complex than standard serial protocols, adding expense for developers, but the complexity brings advantages.

Taking Advantage of TCP/IP

Ethernet/IP uses all the transport and control protocols used in traditional Ethernet, including TCP, IP, and the media access and

Where to Go for More Information

Web Sites

EtherNet/IP, DeviceNet, and ControlNet
Foundation Fieldbus HSE and H1
Profinet and Profibus

Books and Papers

Industrial Ethernet in 90 Days or Less, by John Rinaldi, Real Time Automation.
Ethernet: The Definitive Guide, by Charles E. Spurgeon, O’Reilly & Associates, March 2000.
10 Things to Consider Before Installing Industrial Ethernet, by George Thomas, Contemporary Controls.

signaling technologies found in off-the-shelf Ethernet interface cards. Building on these standard PC technologies means that Ethernet/IP works transparently with all standard off-the-shelf Ethernet devices found in today’s marketplace. It also means that Ethernet/IP can be easily supported on standard PCs and all of their derivatives. Even more important, basing Ethernet/IP on a standard technology platform ensures that Ethernet/IP will move forward as base technologies evolve in the future.

The groups supporting Ethernet/IP plan to ensure a comprehensive, consistent standard by careful, multivendor attention to the specification and through certified test labs. Certification processes modeled after the programs for DeviceNet and ControlNet will ensure the consistency and quality of field devices.

Interoperability?

Is there interoperability among application layer protocols? Basically, no. Foundation Fieldbus HSE devices will not talk to Modbus/TCP devices, nor will EtherNet/IP devices talk to IDA devices. However, the situation is not quite as bad as it may appear.

First, all these protocols use TCP/IP and Ethernet. So ISO layers 1–4 are already agreed upon. That’s a huge step forward. Second, they all can coexist on the same wire at the same time. Third, there’s nothing to prevent vendors from making devices that support more than one protocol. Some vendors do, and some devices can automatically recognize which protocols are being used on the network. Furthermore, a device can use any number of protocols simultaneously. Finally, objects that define the relationships between protocols will be adopted by standards organizations. The issue of low-level device compatibility will slowly fade as the technology matures, and standards will compete at higher levels–system–level integration of objects and profiles such as those found in Profinet, IDA, and OPC.

But in the meantime, always be certain what protocols are supported by the I/O and smart devices you purchase.

Editor’s Note

Portions of this article were taken from Industrial Ethernet: A Pocket Guide, by Perry S. Marshall ©2002 ISA Press.

SIDEBAR:

Embedding Industrial Ethernet in Your Product

Once you decide that your products must support Ethernet, the big question becomes, how can I get it done and get it done quickly?

Do you have excess engineering resources to commit to the project? Can or should your existing opportunities and customers for Ethernet be put off until next year? Is your engineering staff intimately familiar with TCP/IP and the specifications for EtherNet/IP, Profinet, or Foundation Fieldbus?

Once you’ve made a decision to add Ethernet connectivity, you have several ways to proceed. The choices vary in time to market, supportability, resource requirements, and price.

• Use an Off–the–Shelf Serial Gateway. An off-the-shelf serial gateway is the most costly approach but offers the fastest time to market. To implement a serial gateway, your product must support a common serial communications protocol, such as Modbus, DF/1, or even dumb ASCII. Data from your product will transfer to the gateway at serial baud rates (1200–19200).

Other than time to market (typically a week or two), there are few real advantages to this approach and many negatives. Gateways require an additional footprint, they can be costly, they rarely support messaging characteristics needed for your data, and the gateway vendor name appears on the screens of network configuration tools.

Most important, your data are represented as generic data. The characteristics of your device are lost to the network. This option is attractive only if you expect low-volume requests for EtherNet/IP connectivity.

• Add-On Daughterboards. An add-on daughterboard lets you add EtherNet/IP to your product and achieve fast time to market. An add-on PCB plugs into your device internally (usually TTL-level communications) and acts as a serial gateway with less cost and sometimes more benefits.

Internal daughterboards bring with them all the disadvantages that gateways have unless the vendor can customize the data representation and messaging characteristics. If your product contains a spare communications port and you can get a customized implementation, this approach is great for medium-volume applications (i.e., a few hundred units per year).

• Design in an IC Containing EtherNet/IP. Several companies offer microprocessor units (MPUs) with embedded Ethernet and TCP/IP protocols. If you’re building a new device from scratch and the MPUs fit your budget and taste, they can be an attractive option. You control the object-model implementation and messaging characteristics without the speed constraints imposed by gateways. And you can create a similar solution by combining a system-on-a-chip with the next option.

• Purchase a TCP/IP and Application Layer Stack. For a one-time charge with no royalties, these stacks offer you the latest capabilities while giving you absolute control of your development, network presentation, and implementation. This approach requires a dedicated resource to integrate your MPU, TCP/IP stack, EtherNet/IP stacks, and OS (if needed). The advantages of this approach are relative low cost and tightly integrated firmware. Disadvantages include the cost of dedicating internal resources to this effort and the resulting time-to-market issues.

• Custom PCBs. Contract engineering companies can provide you with a custom communications card for your application, usually in 90 days or less, depending on the complexity of your application.

• Do It Yourself. The costliest, lengthiest, and riskiest approach is to build it yourself. While this is admittedly what you’d expect to hear from a company whose business is selling custom networking hardware and software, the facts still speak for themselves.

Like all complex protocol implementations, there are nuances to the specifications that are not readily discernable. The internal resources required for a do-it-yourself project are usually costlier than using an outside resource or buying a component. The risk of missing seemingly innocuous nuances and missing a ship date is much higher than the cost of outside software or engineering.


John Rinaldi is President, Real Time Automation, 2825 N. Mayfair Rd., Ste. 11, Wauwatosa, WI 53222; 414-453-5100, fax 414-453-5125.

Reader Comments

I recently read the article "Distributing Control with Ethernet and TCP/IP" published on your website. While the article was generally informative, I found it misleading while describing the operation of IP protocols and Ethernet.

In the section titled "Switches Segment the Network."
Switches do not "segment" the network. Routers or gateways do. On Ethernet networks, there are two kinds of packets; directed and broadcasted. Broadcasted packets operate similarly to hubs in that one packet in, the packet is sent out every port. For directed packets, the switch receives the packet and "forwards" it to the appropriate port. The Ethernet "segment" is also called the "local" network or the network in which all local devices are connected to without having a router "route" the packet. Traditionally (before switches), routers were used to "segment" the network. This contained traffic and provided a loosely interpreted isolation of the network segment.

Switches do not provide determinism. Switches improve determinism in two ways. Because switches manipulate the traffic, a full duplex Ethernet connection is possible. This creates the capability for the client to be receiving and sending packets to the network; this prevents the possibility of a receiver/transmitter collision (one of the primary causes of deterministic failures). The other improvement in determinism is the switch (there is no such thing as an "intelligent" switch) forwards directed packets only to the intended receiver; thus minimizing the amount unnecessary traffic bogging down the network clients.

Determinism may also be undermined if a backbone (a primary communication pipe) becomes saturated. In this event, packets are dropped. This scenario is unchanged between a switched or hub network.

Featured switches may provide an additional capability called flooding controls. In the event of a "broadcast storm," the result of a network PHY failing, a traditional network would become polluted with errant broadcast. The switch may limit how many of these packets over a period of time the device may transmit hence reducing the impact over the network segment. Switches may not provide this feature and as a result, the broadcast storm problem in a hub scenario remains unchanged in a switched network.

Switches allow higher bandwidth a "better" predictability when a packet will arrive and it’s probability of arrival. The addition of a switch does not just eliminate the problem.

In the section titled "TCP/IP Messaging Services."
UDP is a "data protocol" residing over the IP (Internet Protocol). TCP (Transmission Control Protocol) is also a data protocol over IP. The mail example for UDP missed some of the key points that represent UDP.

UDP is described with the following features:

  • Informal data transmission "datagrams."
  • Arrival is not guaranteed.
  • Order of reception is not guaranteed.
  • Messages have a limited length.
  • Bidirectional communication.
  • Socket model preserves packet boundaries.

Quoted From RFC768:
"This protocol provides a procedure for application programs to send messages to other programs with a minimum of protocol mechanism. The protocol is transaction oriented, and delivery and duplicate protection are not guaranteed. Applications requiring ordered reliable delivery of streams of data should use the Transmission Control Protocol (TCP) [2]. "

TCP has the following features:

  • Coordinated "private" data transfers.
  • Guarantees arrival.
  • Guarantees order reception.
  • Order is guaranteed.
  • Stream size is not limited.
  • Bidirectional communication.
  • Socket model is a stream model (packet boundaries are not distinguishable).

RFC793 describes the operation of TCP.

After "Automation’s Special Needs."
The "data protocol" as the article mentions after the "Automation’s Special Needs" is simply an organization of bytes that are transmitted over a UDP or TCP socket; or even as "raw" Ethernet packets. The content does not clear disseminate that a TCP socket could just as well be used to port 23 (Telnet) instead of these data protocols mentioned.

At Opto 22 and my participation in the Open Source community, I have fielded many questions and problems based on misconstrued ideas about networking, terminology, and its data protocols. The publication of an article that dramatically oversimplifies and misrepresents the network operation might sell these ideas to control/automation engineers in concept but, when they run into problems, they find out the hard way the issues they didn’t know about.

TCP, UDP, IP, and Ethernet are absolutely powerful tools for this industry. It’s extremely sophisticated and it’s a burgeoning industry. It’s important that all information be as correct and clear as possible to allow those readers to make fair and intelligent decision why it’s a medium to use.

Bryce Nakatani
Opto 22 Engineering


Response to letter by Bryce Nakatani

Dear Editor:

First, on the subject of segmenting networks: Yes, switches do segment networks by directing traffic to its intended destination, and as you point out, can block garbage from faulty devices, for example. Switches, routers and bridges all segment networks – each on a different level.

On the subject of determinism: Switches generally make determinism possible. The allotted space for this article did not permit an exhaustive treatment of this subject, nor was there room to delve into the inner workings of UDP and TCP/IP.

A more complete discussion of these topics can be found in my short book "Industrial Ethernet: A Pocket Guide" (ISA Press, 2002) and in several other sources cited in the article. Of course one can always get the entire picture by reading the IEEE specs, if one has the patience to wade through them.

Best,

Perry Marshall

"Originally appeared in Sensors Magazine"