EASInet - IBM's Contribution to Scientific Networking in Europe

Peter Streibelt
C2,1
D-68159 Mannheim

27th of June, 1992

Document Number STR92-SJ01 

Abstract

In November 1987, IBM launched the EASI programme which establishes partnerships with leading European academic and research organisations to foster supercomputing education and research.

EASInet was a logical development of the EASI programme to permit the EASI partners to exchange information on supercomputing techniques and to share in joint research where appropriate.

This paper will describe the multiprotocol environment, connectivity and routing, the concept used for central management and operation, and the convergence with other existing networks.

We are addressing the issues in integrated wide area networks, local and metropolitan networks and high performance attachment of computing systems, and demonstrate that EASInet is an important component towards a future european high speed backbone network. 

Table of Contents

Abstract

Figures

Introduction

Data Networking

  • American Internet
  • European Networking
    •  EASInet
  • The EASI programme
  • Objectives
  • Networking Environment
  • Technical Solution
  • The IBM 973x backbone
  • SNA/SNI Connectivity
  • X.25 ISO/OSI support within EASInet
  • The NSFNET T1-Technology
  • The T1-Link
  • The CYLINK 4201 ACSU
  • EASInet Router Technology
    •
  • The Network Management Tools
  • Transmission of OSI Data Traffic
    •      Collaboration with other networks
  • Conclusion
    •

    Figures

    1. Geographical map of EASInet
    2. The IDNX backbone of EASInet
    3. The german EASInet connections
    4. Details of the T1-Link
    5. Infrastructure at EASIgate
    6. Some statistical data
    7. Configuration for ISO CLNP support
    8. EASInet as of 06-03-92

    Introduction

    The vision of a pervasive high performance international network operating at magabit speeds and serving a wide range of research interests is becoming a reality, however, the European international connectivity is lagging behind. The main reason for the slow progress is the high cost of international circuits in Europe. Other barriers are technical, commercial, organisational and political, like the absence of pan-European suppliers of leased lines, the absence of wide collaboration between industry, government, PTT's and the research and educational community.

    We will give a short introduction on some networking related highlightes in America and Europe, before we demonstrate EASInet, an important component towards a multiprotocol backbone for Europe. 

    Data Networking

    American Internet

    Wide Area Networking (WAN) is considered to have started in the U.S. in approximately 1975 with the ARPAnet. This was a network set up by the Defense Advanced Research Projects Agency (DARPA) and linked sites involed in advanced research funded by the American Department of Defense.

    The technology used in DARPA includes a set of network standards that specify the details of how computers communicate, as well as a set of conventions for interconnecting networks. Commonly referred to as TCP/IP it supports the interconnection of many disparate physical networks and makes them function as a coordinated unit, called an internetwork, or internet.

    Most of the domestic Internet is supported and coordinated by Federal agencies, in principal the Defense Advanced Research Projects Agency (DARPA), the Department of Energy (DOE), the National Aeronautics and Space Agency (NASA) and the National Science Foundation (NSF).

    The National Research and Education Network (NREN) component of the High Performance Computing and Communications (HPCC) program dramatically expands and enhances the U.S. portion of an existing worldwide infrastructure of interconnected computer networks. In particular the NSF backbone network (NSFNET) is running at 45 Megabits per second already.

    Advanced Network Service, a consortium of IBM Corporation, MCI Communications Corporation and MERIT Inc., has been founded to enter the business of data networking, initially for the academic and the research community.

    The new not-for-profit organization manages and operates the federally-funded National Science Foundation Network (NSFNET) backbone, under subcontract to MERIT Inc., as well as provides a broad spectrum of networking services to researchers and educators in universities, federal laboratories and the private sector.

    European Networking

    Many European countries have national data networks for use by their academic and research community. These networks are supplied by the national PTTïs and do not exceed 64kbps since the operational costs can not be provided by the universities and research institutes attached to the network.

    The European Academic and Research Network (EARN) which was sponsored by IBM in 1984, was proving to be a success. Even old-fashioned, EARN was very robust and made it relatively easy for other countries to join and quickly obtain a reliable connection at a reasonable cost. About 50% non-IBM machines attached to EARN and their users participated in the mail and file transfer services, these older technology provided.

    The Cooperation for Open Systems Interconnection Networking in Europe (COSINE) was responsible for setting up an Interim X.25 Infrastructure (IXI), which has been providing an X.25 service via 64kbps lines, connecting various countries together which have poorly developed infrastructure.

    RARE, is the Association of European Research Networks and the contract holder for the COSINE implementation phase. At its Council of Administration (CoA) meeting in Blois in May 1991, it established a Task Force to examine the possibility of creating an Operational Unit to take responsibility for the provision of pan-European backbone and value added networking services.

    The TCP/IP (Transmission Control Protocol / Internet Protocol) protocol suite, as used in the American Internet, has a substantial influence on the growth of the European Internet. Since this growth has not been officially planned, it has to rely on very informal management agreements. The IP coordination across Europe is handled by RIPE (Reseaux IP Europeens).

    EASInet

    The EASI programme

    The EASI (European Academic Supercomputer Initiative) programme which IBM launched in November 1987, established partnerships with 18 leading European academic and research organisations in 9 countries to foster supercomputing education and research.

    The program contained the following four major components: 

                       EASI Installations

Country              City                      Institution
_____________________________________________________________________
Austria              Vienna                    University
Belgium              Leuven                    KUL
France               Lyon                      IN2P3
France               Montpellier               CNUSC
France               Paris                     CEA
France               Paris                     CIRCE
Germany              Aachen                    RWTH
Germany              Braunschweig              Technical University
Germany              Darmstadt                 GSI
Germany              Hamburg                   DESY
Germany              Karlsruhe                 KfK
Italy                Bologna                   CINECA
Italy                Rome                      University
Netherlands          Amsterdam                 SARA
Spain                Barcelona                 FCR
Sweden               Skelleftea                UMEA
Switzerland          Geneva                    CERN
Switzerland          Zuerich                   ETH
    EASInet was a logical development of the EASI programme to permit the EASI partners to exchange information on supercomputing techniques and to share in joint research where appropriate.

    Figure 1. Geographical map of EASInet
    *** Please see graphic ***
    To coordinate the interests of the EASI sites and to decide on policy and general rules for the network an "EASInet Project Committee" (EPC) was formed. It consists of the representatives of five EASI sites, one representative of IBM, one of GMD and to intensify cooperation with other European networking organisations, one representative of the RARE Council of Administration (RARE - CoA). The EPC has established a lot of cooperative agreements with national and international European networking organisations. For many of them EASInet transports US-traffic or establishes European connectivity. Also many line sharing arrangements were made to create a backbone with higher speeds than 64Kbps.

    To avoid a bottleneck on the line between America and Europe, IBM ordered a T1-link (1.544Mbps) from the begining. Since this doubled the existing bandwidth for scientific networks between Europe and the U.S. it is a very attractive facility for other networks to collaborate with EASInet and improve the efficiency of scientific networking, EPC established a recommendation for the use of EASInet: "whoever applies for access to the T1-link should try to contribute to existing bandwidth by either increasing the bandwidth of used lines or by merging existing lines with EASInet-lines". This led to a network with much better international connectivity and much more bandwidth than was available in Europe before. In fact, EASInet established the first international European backbone for research and science that extended to multiples of 64Kbps in bandwidth. 

    Objectives

    In the context of the considerations for the realization of the network it was obvious that no EASI partner could be interested in enlarging the number of its already existing network connections by one more. This would unnecessarily increase the complexity and the effort to operate the network infrastructure. Instead there was a tremendous demand to increase existing bandwidth or at least consider the integration in a future networking concept.

    The efforts undertaken by IBM, to find a European carrier or any other vendor who would collaborate with EASInet to contribute to a "European Broadband Link" at that point in time, failed. So the European networking service providers were left alone in trying to set up a simpler, more cost-effective and more efficient networking infrastructure for research and education.

    It was also clear from the begining, that it would not be sufficient to provide the EASI partners with physical lines only. To guarantee connectivity it was required that IBM established a central network management and started a well coordinated and careful planning of EASInet as an additional offer. GMD, the German National Research Laboratory for Computer Science, was subcontracted to help design, implement and operate the network. 

    Networking Environment

    The goal of EASInet is to provide different applications to the end user using different protocol stacks. To support these different protocols in one backbone scenario EASInet made a lot of investigations with representatives of the 18 EASI sites to understand the special needs and requirements of each particular site. The outcome of this was the following:

    Technical Solution

    In general EASInet is designed to support different communication protocols (like TCP/IP, X.25, SNA, CLNP, DECnet, ...) on the same underlying infrastructure. Concept and design of this infrastructure depends partly on the facilities and services European PTTs provide to customers in the scientific community. Besides these technical issues the political conditions in the European Networking Communities led to some special agreements and solutions. Reflecting this circumstances EASInet implemented mainly two different kinds of multiplexing.

    The IBM 973x backbone

    Currently the IDNX backbone consists of fourteen IDNX boxes distributed throughout Europe.

    Figure 2. The IDNX backbone of EASInet
    *** Please see graphic ***
    Each IDNX box is used to separate communication protocols on international communication links. This enables the network provider to guarantee bandwidth for different purposes.

    In general, time division multiplexing can be done in three different ways. First, using a kind of split modems (DSU/CSU) which allocate bandwidth on a permanent basis using fix granulation (i.e. 9600 baud or 64000 baud channels). Fix granulation leads usually to a certain amount of internal fragmentation, which can not be tolerated because of European communication tariffs. Second, using a statistical multiplexing technique, which utilizes the bandwidth on demand. This is the optimal way to communicate but equipment supporting these multiplexing technique is not available for such line speeds at a reasonable price.

    Finally, the IDNX box uses a third kind of bandwidth allocation scheme called 'dynamic bandwidth allocation'. In general, during the lifetime of a connection (using one protocol, between two transmission facilities) the amount of bandwidth is fixed. The interesting point here is, that the granulation of the allocated channels is not fix. That means, we can assign in a flexible way bandwidth to different protocols and/or communities as requested, using 'dynamic bandwidth allocation'.

    This feature and the ability to manage the resulting IDNX backbone remotely in an efficient and sophisticated way were the main points in choosing IBM 973x multiplexer. Last but not least the IDNX boxes are able to support digital voice transmission, which become more and more importance in Europe.

    The fourteen IDNX boxes build up one network. This means, each IDNX has knowledge about all available (connected) IDNX's somewhere in the network. This leads to a point of view as a whole integrated level 1 network. It is assumed that each IDNX has a unique number assigned, so that we can identify each backbone EASI site by an IDNX Node number (IN).

    Here the actual list of IDNX nodes: 

    IS                       IN         connected to       managed by
-----------------------------------------------------------------------
CEA Fontenay             N1         N2, N3             CEA
CEA Paris                N2         N1, N3             CEA
CEA Saclay               N3         N1, N2, N4, N5     CEA
CEA Grenoble             N4         N3, N5, N6, N141   CEA
CEA Cadarache            N5         N3, N4             CEA
CEA Pierrelatte          N6         N4, N7             CEA
CEA Marcoule             N7         N6                 CEA

CIRCE Paris              N33        N133               CIRCE/GMD
SARA Amsterdam           N131       N141               SARA/GMD
CNUSC Montpellier        N133       N33, N141          CNUSC/GMD
CERN Geneva              N141       N4, N131, N133,    CERN/GMD
                                    N149, N233, N249   CERN/GMD
GMD Bonn                 N149       N141               GMD
IN2P3 Lyon               N233       N141               IN2P3
DESY Hamburg             N249       N141               DESY/GMD
    To allocate bandwidth between two (data) ports of two (usual) different IDNX boxes the network manager just defines the amount of bandwidth together with the source port and destination port of the involved IDNXs. Each IDNX on the way from source to destination determines the amount of resources needed (internal bus capacity, spare bandwidth pool, ...) and if possible made the necessary allocations. To communicate with each other for internal administration purposes the IDNX uses its own protocol, called SCLP (Signal Channel Link Protocol). SCLP based on the recommendation SS#7, but its specification is not published. The amount of bandwidth used for IDNX internal communication varies between 6400 and 16000 baud, depending on the kind of physical link leased from the local carrier.

    To manage the IDNX backbone it is necessary to have some detailed information about the actual status of the backbone nodes and the occurring events. Each IDNX node has two types of event logs:

    Consequently there is one designated IDNX node which collects all network significant events in the EASInet IDNX backbone. In our case it is the IDNX node N149 at GMD Bonn, the Network Operation Center (NOC). At the NOC we have installed additional equipment, namely IBM Netview/PC, IBM NetView R3 and N.E.T. ALERTMON to monitor the IDNX backbone automatically, convert the IDNX events to SNA NMVTs (Network Management Vector Transport) and display them on a centralized NetView NPDA (Network Problem Determination Application) panel on the NOC host.

    Additionally GMD NOC has installed the EOC, Enhanced Operator Console, an IBM PS/2 system providing reporting and statistical data gathering tools to analyse the IDNX backbone event and status information in a detailed way. This helps to recognize as soon as possible physical link problems in correlation with reliability and availability degradations of specific backbone connections. 

    >SNA/SNI Connectivity

    Those EASI sites who need connectivity for their mainframes running SNA (IBM's Systems Networking Archtecture) are connected via the SNI technique. SNI (SNA Network Interconnection) allows to manage large SNA networks by splitting them into many small networks or to connect other autonomous networks. Within EASInet the 'back-to-back' technique of SNI has been implemented. With 'back-to-back' the participating SNA networks remain nearly autonomous. They only need to agree to a few definitions of a common 'Null-net' to which they are connected via one or more gateways. The gateway function is part of the SNA access methods on the IBM mainframes.

    The SNI network of EASInet itself has gateways to other SNI networks within Europe e.g. EARN (European Acadamic Research Network) or AGFnet (Network of the German National Research Centers and Universities).

    In Germany the SNI network of EASInet is implemented on top of X.25, this means that all the German EASI sites are connected to WIN and via SNA ports in the IBM 3745 of DESY, Hamburg, to the international IDNX-backbone. 

    X.25 ISO/OSI support within EASInet

    All German EASI sites are connected to WIN. WIN (WIssenchaftsNetz) is an X.25 network, based on '84 standard, offered by DFN (Deutsches Forschungsnetz) with speeds up to 64KBit/s. Dependent on the need and the availability, the German EASI sites are running OSI applications via WIN. Mostly used OSI applications are those supporting the ISO/CCITT-standards X.400 for electronic mail and X.29 (X.3/X.28) for line oriented dialogue.

    Primarily the X.25 infrastructure of WIN is used to get connectivity for SNA/SNI, TCP/IP and DECnet. International X.25 connectivity for the EASI sites - if needed - is provided via the X.25 ports of the IDNX backbone, e.g. on the link between DESY and CERN.

    Figure 3. The german EASInet connections
    *** Please see graphic ***

    The NSFNET T1-Technology

    The IBM designed packet switch, also known as the Nodal Switching System (NSS), provides access to the NSFNET backbone, "switching" between backbone nodes, intelligent routing and network management. The NSS is a high-throughput, low-delay packet switch that combines IBM products in a modular design that is extendable, serviceable and fault tolerant.

    This concept has proven its quaility and rich functionality now for more than three years in the U.S. and nearly two years in Europe.

    To achieve this, the NSS design connects multiple RISC-processors based on IBM RT/PC's, (IBM 6151 and IBM 6150) connected to a dual token ring. This dual token ring guarantees the requested fair access method to the common network media. Although a NSS consists of more that one processor it looks like as one functional unit from the outside.

    The processors attached to the dual token ring are divided in the following categories:

    For a detailed description of the NSFNET technology the reader may look at the NSFNET proposal dated November 1987 'MANAGEMENT AND OPERATION OF THE NSFNET BACKBONE NETWORK' which shows the original concept of the NSFNET T1 backbone network in the U.S.

    The T1-Link

    On march 15, 1990 IBM Europe announced the largest and fastest data communication link between universities and research organisations in the United States and Europe.

    Information coming to Europe from the U.S. enter EASInet at the European Laboratory for Particle Physics (CERN) in Geneva, Switzerland, which is an EASI partner. The American gateway to NSFNET is the Cornell National Supercomputer Facility at Cornell University in Ithaca, State of New York.

    The line ends at EASIgate, a token ring infrastructure with all kinds of processors to provide support for EASInet connections and the T1-Link to NSFNET.

    The CYLINK 4201 ACSU

    The connection of the physical T1-Link at CERN and Cornell is done via the 4201 Advanced Channel Service Unit (ACSU), a stand-alone D4 or Extended SuperFrame (ESF) channel service unit. It performs the following functions: The machines at CERN and Cornell accommodate ESF framed input, using Alternate Mark Inversion (AMI). On the network side, ESF framing and AMI network coding are selected.

    The NOC can remotely configure and diagnose the ACSU and obtain internally stored alarms and performance data by means of the CYLINK Netwoek Managment System (CNMS). The available RS232C port operates asynchronously at 1200 bps and is a dial-up connection.

    A detailed view of EASIgate and the physical T1 circuit is shown.

    Figure 4. Details of the T1-Link
    *** Please see graphic ***
    Figure 5. Infrastructure at EASIgate
    *** Please see graphic ***

    EASInet Router Technology

    At CERN one processor called Split Exterior Packet Switching Processor (Split-E-PSP) was placed as an extension of NSS-10 in Cornell to offer an access point to Europe with the same functionality as any other E-PSP processor direct connected to a NSS. This split-E-PSP called psp-10-s1-2 is fully managed by MERIT Inc., the NOC which manages NSFNET, and controlled by the RCP Routing Control Processor of NSS-10. The psp-10-s1-2 is used as the access point between EASInet and NSFNET to offer packet switching functionality as well as the exterior gateway protocol EGP. EGP is used to exchange routing information between NSFNET and EASInet.

    The EASInet engineers have chosen the token ring as the media to interconnect NSFNET and EASInet. Attached to this token ring are routers managed by EASInet.

    These routers offer connections to the CERN infrastructure and to other regional networks that are connected via serial lines to EASInet routers directly.

    To benefit from the NSFNET technology, EASInet uses IBM RT/PC's as routers as well.

    Due to different user requests and different line speeds that are feasible, the NSFNET model has been slightly modified to achieve the best for the European situation:

    Currently a single token ring is sufficient for our needs, because EASInet has less traffic than NSFNET.

    Figure 6. Some statistical data
    *** Please see graphic ***

    The Network Management Tools

    Monitoring and managing complex networks involves acquiring and processing information from a large number of different sources.

    XGMON is a network management facility, developed for the NSFNET network backbone. Our NOC is using XGMON to manage the IP part of EASInet, since the program offers a Simple Network Management protocol (SNMP) client. In the IP world, SNMP is available with most vendor products and therefore allows the management of a heterogenous networking environment.

    The SNA related network management information in EASInet is collected and processed via IBM Netview. The program has been discussed in detail in a former Systems Journal (Volume 27, Number 1, 1988).

    The network management service for X.25 is basically provided by the PTTs so that GMD is not much involved. The problem with X.25 network management information is that the data which are provided is not unique and therefore very hard to process. The reason is the different vendor equipment being used.

    Within EASInet we currently develop a network management platform for a multiprotocol environment based on the ISO standards Common Management Information Services and Common Management Information Protocol (CMIS/CMIP). This prototype will collect information from the IDNX boxes, IP routers, 37xx's and X.25 switches to perform the actual processing.

    Transmission of OSI Data Traffic

    The international exchange of ISO CLNP packets was demonstrated between end systems at the NSFNET network operations center in Ann Arbor, Michigan and the EASInet network operations center in Bonn, Germany, in August 1990.

    Figure 7. Configuration for ISO CLNP support
    *** Please see graphic ***
    The current NSFNET prototype OSI implementation is intended to provide wide area connectivity between OSI networks, including networks utilizing the DECnet Phase V protocols.

    As NSFNET today is primarily a TCP/IP network, the OSI implementation is designed to coexist with the operational IP NSFNET. Both DoD IP and ISO CLNP packets are forwarded together through the same packet switches and links. CLNP packets are transmitted "native" on the links; they are not encapsulated within DoD IP. Dynamic routing is supported for both protocols, providing automatic rerouting in case of an outage.

    The NSFNET OSI implementation supports the ISO Connectionless Network Protocol (CLNP, ISO 8473). This is a datagram protocol, similar to the DoD Internet Protocol (IP), providing best effort delivery of packets. Each CLNP packet is examined by the backbone and forwarded along the best path toward its destination.

    The implementation supports the transmission of CLNP packets over T1 serial links, IEEE 802.3 CSMA/CD LANs, and IEEE 802.5 Token Ring LANs.

    Routing and address resolution between NSFNET, End Systems and other Intermediate Systems is provided by the End System-Intermediate System routing protocol (ES-IS, ISO 9542). This protocol provides functionality similar to the Address Resolution Protocol (ARP) and the Internet Control Message Protocol (ICMP) Redirect functions used in the TCP/IP protocol suite. Routing internal to the NSFNET backbone is based on the ISO IS-IS intra-domain routing protocol (ISO DIS 10589).

    Interdomain routing between the NSFNET backbone and other routing domains is currently static, awaiting the development of OSI interdomain routing protocols.

    Collaboration with other networks

    As mentioned before, EASInet has reached some agreements with other networking organisations in Europe by sharing lines which provide a multiple of 64Kbps. A number of mutual backup agreements exist for the T1-Link as well.

    Figure 8. EASInet as of 06-02-92
    *** Please see graphic *** 
     

    Remarks: This foil has a multiple purpose: 
    1) The red frames (boxes) are the EASI sites 
    2) The black solid lines demonstrate our 3-year 
    commitment for 64Kbps connectivity 
    3) The green lines show the agreements in 
    bandwidth we have reached with other organisations 
    4) A description of bandwidth splitting in terms 
    of protocols multiplexed via the IDNX could be 
    found as well 
     
     

    The following table shows the line sharing agreements which exist today: 

                       EASInet Cooperations

Line from       Line to        Bandwidth       Partner Network
_____________________________________________________________________
Stockholm       Amsterdam        192Kbps       NORDUnet
Amsterdam       Geneva           256Kbps       HEPnet, SURFnet, EUNET
Hamburg         Geneva           768Kbps       HEPnet, DESY
Montpellier     Geneva           256Kbps       CNUSC, EARN
Lausanne        Geneva          2048Kbps       SWITCH
Paris           Geneva           256Kbps       CEA
Lyon            Geneva          1024Kbps       IN2P3

    Conclusion

    EASInet, the second IBM initiative in data networking for Europe, has improved the national and international networking connections for the EASI partners and other European networks. The authors believe that EASInet has stimulated the sort of collaboration in european data networking towards a pan-European backbone. The heterogenous environment we have implemented and in operation today for the research and education community could be extended to support commercial customers. We have proven that Europe has the skills, the industrial strength, and, we hope, also the will, to continue to make progress with a solid pan-European infrastructure for data networking. There is still a lot of work to be done to improve the understanding of the "problems" of european data networking among Europe's politicans, PTT's, network providers, industrialists and network users.