The
Next Generation of IP - Flow Routing
Dr. Lawrence G. Roberts
Founder, CTO Caspian
Networks
SSGRR 2003S International Conference, L’Aquila Italy, July 29, 2003
Abstract: For the last 33 years IP routers have not changed,
they still support only “best effort” traffic. However, the bandwidth available
to people has been increasing rapidly with the advent of broadband access. The
result is that many new services are now desired that require far better QoS
than “best effort” IP can support. Also, with broadband, the problem of
controlling the total usage and carrier expense has become important. Thus, it
has become critical to improve both the delay performance and the control of
bandwidth for IP service, much as was accomplished in ATM. Also, call rejection
for high bandwidth streaming services like video is required instead of random
discards if quality is to be maintained. All these problems can be solved with
no change to TCP/IP by routing flows rather than packets. This requires keeping
some state information for the duration of the flow, but this information can
be captured on the fly as the first packet goes by. This permits an IP flow
router to achieve all the capabilities of an ATM switch plus some, but without
the call setup delay and at a lower cost than a conventional IP router.
Summary of the IP QoS Issue
Problem: IP routers historically have never supported Quality of Service (QoS)
due to the fact that they only look at packets and do not keep any state
information on individual flows. A flow can be defined as the stream of IP
packets between a particular source IP address and port going to a unique
destination IP address and port, all of which packets are using the same
protocol. An individual flow might be a voice call, video call, file transfer,
or web access. For TCP flows, the network controls the flow rate through
discarding packets. For UDP flows, the user controls the rate but the
introduction of delay variance or packet loss in the network can be a major
problem. The problem then is to design an IP router that can support rate
guarantees, delay guarantees, and eliminate packet loss for flows of any size
and still maintain high line and trunk utilization.
Prior Work: ATM attempted to solve the QoS problem and much was
learned during the design of ATM, but due to the long, complex software flow
state setup process ATM could not be used for short data calls. Another
approach that is being tested is to use MPLS with class of service (CoS)
queues. MPLS however cannot support QoS for individual small flows, cannot
provide delay guarantees, and cannot reject new flows to protect current flows
from packet loss. Current IP and MPLS routers attack the QoS problem by under
loading the network but this increases the cost of IP as much as 10:1. Instead
of wasting network capacity, some recent researchers have looked at improving
IP routers through the use of some flow state information. (1, 2, 3, 4, 5, 6)
This work has been largely theoretical but has concluded that using some flow
state information inside an IP router could substantially improve the
throughput and QoS.
The Caspian Flow Router Design: This paper describes the design and results
actually achieved in a real IP flow router product that can scale to extremely
large capacity and can support millions of flows per port. Full ATM QoS
capability can be achieved at very high levels of utilization and many other
benefits have resulted from flow state information. The important conclusion is
that flow state can be incorporated into an IP router, without modifying the IP
protocol, providing all known QoS features, without increasing the cost, while
permitting the utilization to be increased to over 80%.
History of the Current “Best Effort” IP Routers
When
I started the ARPANET in 1969 (7) there was so little memory available on the
original IMPs (4 K bytes for both program and data) that it was impossible to
consider storing any information on each flow in the network. Thus, it was
clear that “flow state” could not be used due to severe memory constraints.
State information had historically been used in telephone switches yet with the
advent of packet data communication, I realized that the time required to set
up the state information across a circuit path was too lengthy and required far
to great an amount of computer resources to support short data transfers. In
most cases the call setup for a data transfer (averaging 14 packets per
transfer) would be more expensive than the actual data transfer. The original
packet protocols, NCP on the ARPANET and X.25 in the early carrier networks
(8), saved each packet on each hop until it’s receipt was acknowledged so that
data could be transferred without loss or error The packet flow forged its path
across the network at full line speed and only a short time was required to
retransmit the packet in event of error over the last link. For the high error
rate telephone line used then, this was the fastest way to transfer a lossless
file. During the original pre-Internet MIT-SDC network experiment I conducted in
1965 (9), the packet length was carefully constructed to minimize the total
file transfer time including retransmissions. The Internet started with this
packet retransmission approach although the optimal packet size grew as the
transmission facilities improved so as to continue to ensure the lowest file
transfer time.
1983 to 2003: TCP/IP was fast emerging as the new standard
router—to-router protocol due to its significant efficiencies in eliminating
the trunk by trunk packet storage and retransmit. This only became possible due to the fact that new fiber optical
transmission lines were now error free enough to make end-to-end packet
retransmission optimal. However, given the continuing memory constraints, IP
router designers continued the practice of each router forwarding packets with
no memory of the flow. The user sends
his packets into the network and each router looks only at the destination and
queued the packet for the ‘best’ output trunk currently selected for traffic to
that destination. This scenario requires strict packet ordering of each flow so
as to avoid reordering at the destination. To achieve this, all packets from
all flows, heading for a given location, are sent on the same route, even if
that route is overloaded. If an overload condition occurs, the result is to
drop packets randomly from the queue. For TCP flows this causes TCP backoff and
thereby reduces the load. So long as TCP traffic is the dominant traffic (which
it historically has been), this would sufficiently control the trunk load and
not require UDP traffic discard. However this assumption is no longer
valid. Whenever UDP traffic (voice,
video, etc.) peaks to require more resources than are available given trunk
capacity and in the absence of flow-state information, the only possible
decision is to discard random UDP packets, thus destroying the integrity of all
the voice or video UDP calls in progress The
clear result of this lack of flow-state information is that with conventional
IP routers today, it remains impossible to achieve the same quality of service
(QoS – delay and bandwidth guarantees) that is possible on TDM networks or on
ATM networks.
The Problems with “Best Effort” IP Routers
1.
QoS in the Core: The
concept that over-provisioning of IP trunks can permit “reasonable” QoS is true
in the core where trunk speed is many gigabits/second and there are millions of
flows on each trunk. The law of large numbers works so long as the flows are
small compared to the trunk size and the total traffic in the premium traffic
class never exceeds the available trunk capacity. Delay variance should be very
small for the highest priority class so long as it does not exceed the trunk
capacity. The actual experience with voice over IP in the actual Internet core
does not seem to be as good as this would suggest even though non-Internet
voice only IP networks do far better. There are two probable reasons for this:
a.
Route Changes – IP
routing as it is now applied, selects new preferred routes for all routes in
the general area of a trunk failure. Even if the route that a voice flow is
using has no failure, a nearby failure can cause the voice route to be changed
so as to re-optimize the network. Since this new route will not be the same
distance as the old route, the voice packets can arrive out of order due to the
route change. This causes a gap in the voice received. Since the Internet core
has many routers and trunks the frequency of route changes is far higher than
it would be for a private network and thus this problem is greatly magnified.
b.
Brief Overloads – Even
though most core networks run at 25% load (US average) the only measurements
made today are for 5-minute averages (10). During these 5-minute samples, the
load will vary greatly in a self-similar way to the longer-term variation of
the traffic. Since 5 minute peak traffic on any given trunk varies up to 40%
during peak hour, one can expect that an additional large variation occurs
during any given 5 minute period. Thus, even at 25% average load, the load on a
trunk can peak for a second or so to over 100% causing a brief backup even for
the premium voice traffic.
UDP CAC: No matter what the traffic loading, one problem will
always exist for core routers using the “best efforts” packet-by-packet processing; UDP flows like
video and voice cannot be protected from random discards if the UDP traffic
momentarily exceeds the trunk capacity. When this occurs, today’s IP routers
discard random packets from all of the UDP flows. This causes random holes in
the UDP packet stream; snow in video, noise in voice across all flows in
process (not just misbehaving or new traffic flows). If there is compression or
encryption then the packet loss creates even worse noise and quality loss in
the ongoing UDP flows. This is unavoidable since the router has no knowledge of
flows and thus cannot stop new flows from overloading the trunk.
2.
Routing Restrictions:
Another result of the lack of memory in the early routers was that the router
could not afford to keep much information on the network topology or trunk
utilization. Thus, the practice of calculating and circulating only the
preferred next hop, not the total network topology and load information was
adopted. This required all routers in each network domain to use the same
algorithm (or sharply constrained algorithms) to compute the next hop so as
ensure that no loops occurred. Of course this led to very slow progress in
routing algorithms. Near-equal-cost algorithms, which could provide substantial
utilization improvements, were known from the start but still have not been
deployed very widely. The result of these early memory constraints led to a
router design that became a religion thwarting new routing concepts, rather
than what I had originally intended which was to provide a starting point for
network research. The “net-heads” felt any move toward using state information
was an attack on their religion by the circuit switching “bell-heads”. And the
“bell-heads” knew that true quality of service including delay and bandwidth
guarantees and call rejection could never evolve out of IP Routers as they were
originally conceived and designed.
3.
Failure Recovery:
When the Internet was small there was
no reason to consider how to recover from trunk failures before the failure
because the process was so fast to recompute all paths in the network. This
procedure now takes minutes and has not fundamentally changed because there is
no other routing information available except the preferred path. The result is
that IP today recovers much too slowly and it has been necessary to use SONET
or Optical switchover to a spare trunk (a costly practice) or MPLS fast reroute
(slower than desired and forces the use of MPLS along with its high operating
cost).
ATM: In an attempt to develop a packet-based service that would deliver true
QoS, ATM was emerged in the mid 1980’s. However two major mistake were made
early on:
1.
Round trip software intensive call setup that limited the call setup rate (call per second)
to about 3% of the rate needed for TCP/IP data, thus relegating all data
traffic to be trunked (not switched). This then required packet routers at each
node as well as an ATM switch. This duplication of equipment became a clearly
losing proposition over time.
2.
Fixed Packet Size was
chosen due to the fear that ASIC’s would not be able to support variable size
packets in hardware. The size was chosen small enough to keep the packetizing
delay for voice down to a workable minimum (6 ms). Fixing the size, however,
lost about 20% efficiency for IP data that had packet sizes on average of 350
bytes (or greater) and short burst of packets (10-14 packets per transfer). It
also meant that the processing in PC software due to the frequent interrupts
was too expensive, thus NIC cards with outboard processors were necessary for
all computer interfaces. When compared to cheap dumb Ethernet NIC cards, ATM
NIC cards cost far too much. The
introduction of cheap 100 Mbps Ethernet boards was the final deathblow for ATM
at the desktop.
The
conceptual battle between the IP and Telco people was demonstrated when ATM
adopted a similar concept of round trip flow-path setup used in SS7 by the
telephone network. This decision became the downfall of ATM. It was in fact
voted in by those who wanted the ATM signaling to be fully compatible with SS7.
As the ATM forum first looked at this issue of call setup, I proposed a very
simple forward path setup that would not delay the first packet and thereby
permit short data traffic as well as voice traffic. This concept was soundly
defeated however, by those who felt that this would be incompatible with the
migration of the telephone network and SS7 to ATM. The result was that ATM
could only be used for voice, video and trunks. These trunks would need to use
IP routers to support short data flows like web traffic. Thus, as the volume of
data traffic exceeded voice traffic in year 2000, the use of ATM to trunk IP
became a clearly wasteful proposition. The world quickly moved to the switch
design that supported the majority of the traffic (IP) and people hoped that
somehow IP could be improved to support the same quality of service as ATM.
With voice now a clear minority of the traffic, the hope became that the use of
trunk over-provisioning techniques combined with static pre-engineering of IP
network topology would permit reasonable quality voice.
ATM
had in fact advanced the state of the art in developing important, new robust
techniques for specifying and managing QoS including the extremely innovative
ABR feedback control (11). These techniques proved that packet technology could
achieve the QoS of the voice network and these techniques would clearly be
important if somehow the best of IP and ATM could somehow be merged.
DiffServ (CoS): The response of the router industry to the growing
requests for QoS has been to add a single 6-bit field (the DiffServ Mark) into
each IP packet. This field could in theory specify 64 classes of service, or
queues that could be serviced with different priorities at the output. Since 64
marks has proved to be far too few to specify all the necessary QoS parameters
uniquely (as ATM had previously done), DiffServ marks could only be used by
joint agreement across a single trunk (with exception for a few global
characteristics). Additionally, any of these queues could fill up and
overflow. When this occurs all flows in
that queue (for example video) suffer random packet loss. For voice and video
this practice is a disaster as no one is able to receive quality video. Thus
Diffserv permits what is called Class of Service (CoS) but this is not what
those needing ‘guaranteed service’ such as priority or premium voice, video,
gaming, remote control, or other real time services require. This problem
occurs not only at the edge, but also clearly the same overload can occur anywhere
in the core. Over-provisioning in the core hides the problem at great cost, but
it still can overload resulting in premium packet loss at a lower frequency.
MPLS: When IP arrived at the point
where no clear solution for IP QoS has emerged to address the above issues,
MPLS was proposed as a hybrid circuit—packet merger. MPLS paths (Label Switched
Paths or LSPs) are set up with one of two software protocols (RSVP or LDP) both
as complex and slow as ATM signaling.
As a result:
1.
Nothing was gained in the
ability to use these LSPs for individual flows due to the slow setup speed.
2.
It has no QoS
advantages over IP with Diffserv since the multiple tail dropped queues result
in the same random discard behaviors.
3.
Its primary benefit is
to provide a simple header to permit multiple protocols to be mixed in a single
stream.
4.
Another benefit
claimed has been the ability to route packets to the same destination over two
or more routes which was originally intended to permit better manual traffic
management, The actual result is that this manual management has a high
operational cost and no real gain in trunk utilization.
5.
The final improvement
claimed was the ability to do a “fast reroute” of an MPLS path (100 ms), slower
than SONET yet faster than IP reconvergence. This, however, resulted in a new
dual standard (vendors could not agree on one). The same effect could be achieved with IP at far faster speeds if
near-equal path routing information were distributed as has been planned in TE
extensions to OSPF and ISIS. This alternative could be totally standard,
receivable by any IP router.
Thus
all current communications protocols have either major QoS failings (IP and
MPLS limitations) or cannot support fast data calls (ATM). Therefore no
conventional switches or routers have been able to support the desire of our
next generation, one converged network that can support high QoS and short data
calls with fast error recovery and high availability.
FLOW ROUTING
Realizing
that IP was the only technology adopted by the masses of edge users (supported
by cheap Ethernet interfaces with the flexibility, resiliency and
standards-backing required) and the only technology that could support short
data calls like Web calls at full line rate, IP is clearly the protocol that
can continue to take us forward. The protocol itself does not limit the QoS,
the speed of error recovery, the availability, or the scalability of the
switch. Thus there is no reason to change TCP/IP in order to obtain the goal of
a converged network. In fact due to the worldwide acceptance of IP it would be
unreasonable not to build a new network around IP. The problem has always been
the actual routers’ design. Its history of constrained memory created the
initial design and implementation. For 30 years no one except Caspian Networks
(12) has attempted to change the basic design. A sort of religion developed
that it was heresy to have any state information in an IP router, This religion
was fed by the design required for a conventional router where to keep packets
in order, the only acceptable routing method was to pipeline the routing
lookups using the fastest access memory available, SRAM. This ensured the
packets stayed in order, but since SRAM was vastly too expensive for state
memory, the obvious conclusion in the past was that maintaining state is
neither feasible nor economic.
However,
maintaining state information is essential if one wants to use Connection
Admission Control (CAC) on new UDP flows, guarantee the rate of a flow,
guarantee the delay variance of a flow, route flows independently for load
balancing purposes, or recover immediately from trunk failures. All these are
essential if we are to achieve QoS similar to ATM or TDM over IP. There are two
problems to solve then:
1.
How can the state information be created: The SS7, ATM, and RSVP-MPLS solution is a
roundtrip call setup in software that is clearly too slow for the new real-time
networks. The alternative is to capture the state on the fly as the first
packet traverses the network. This is a forward path setup and can be done in
hardware at full line rate.
2.
How can the state be stored economically: First one must switch from expensive SRAM to the
new PC cache memory, DDRAM that is 75 times less expensive. DDRAM has a
sufficiently high data throughput –yet to maintain the throughput rate, a DDRAM
controller must be used which introduces a 40-60 ns delay variance, far too
high for a pipeline-type of system for packet processing. However, once one
accepts that flow state is to be collected, the basic flow ID of an IP packet
(the five-tuple SA, DA, SP, DP, PR) can be hashed and the flow uniquely
identified. Now parallel processing can be used, dramatically increasing the
processing power available in a chip.
Since the flows are identified, different flows can be assigned to
different parallel processors where no order needs be maintained between
divergent flows. Thus the full DDRAM memory capacity can be used for the route
table, the state table, packet storage, and a large calendar table for
achieving weighted fair queuing per flow. Thus, once one accepts using state,
the memory cost drops so far that state is less expensive to process, not more
expensive. Additionally, packet
processing occurs on the first packet (header) only for a given flow (not all
14 packets as in conventional routers) saving processing resources on a
flow-by-flow basis as well.
Comparison of a Packet
Router and a Flow Router
Packet Router


Flow Router


Unique Characteristics of a Flow Router
Dynamic Load Balancing: Once one has recorded the state information on the
ID, route, time of receipt, and rate of a flow, one can route all subsequent
packets of the same flow on the same route. The route need not be the same
route for all flows to a given destination since each flow can now be kept in
order by being kept on its own route. Thus, with near-equal cost route
information one can distribute the load over all near-equal paths based on the
current load information. It is important to continually measure the load on
all ports so that dynamic load balancing can determine the best path. This load
balancing eliminates all the manual labor of MPLS manual routing and actually
achieves a much higher trunk loading than is possible with static load balancing
due to the rapidly changing traffic load characteristics of IP traffic.
Comparison of Protocols

Fast Error Recovery: When a routing decision
is made for a flow, an alternate, diverse route can also be determined. This is
stored with the state information so that if a card or trunk failure occurs,
the flow can instantly be rerouted in hardware over the new path. Again, this
depends on having multiple near-equal cost paths computed, but given this, each
flow can be assigned the best diverse alternate based on its QoS, rate, and the
current trunk loading. If a path failure occurs, then the first packet in a
flow to arrive at the broken point is turned around and returned to the source
point. This tells the source that that path is bad and the packet and all
subsequent packets can be routed over the alternate path. When other flows
encounter the break, they may have different alternate routes specified since
the actual trunk utilizations across the topology will have varied as different
flows have been set up. Thus no packets are lost after the break is discovered
(10 ms typical) and the flows that were on that path are automatically
re-distributed over many alternative paths, distributing the load. Then, as
these rerouted flows arrive at various output ports, if the total load on the
port is too high, the TCP traffic will be automatically reduced by selective
discard to adjust the total load to the given trunk capacity. The guaranteed
traffic and UDP traffic will not be affected so long as there is sufficient TCP
traffic to absorb the overload. In this way, a network can run at very high
utilization until a failure occurs. Then the lost capacity can be absorbed by
the TCP traffic and the network can remain fully loaded without needing any
spare trunks. The goal of any IP network provider is to recover all voice and
video flows instantly with no impact and give all the excess capacity to TCP
users. A spare trunk would just be lost TCP capacity except during the short
interval of trunk failures. Note also that flows that are on nearby paths to
the break can be left intact without moving them, unlike the current IP
situation where many of these would be moved to rebalance the network at the
cost of out-of-order packets. This is because today IP can only re-optimize the
preferred routes to a destination whereas flow routers can deal at a much finer
grain and re-route actual discrete flows, and not simply large, aggregate pipes
(tunnels) or groups of flows.
Guaranteed Bandwidth: One of important capabilities of TDM, ATM, and Frame
Relay has been the capability to offer actual bandwidth guarantees for
individual flows like voice, or for groups of flows like a VPN across the
network. IP packet routers have no capability to offer individual flow
guarantees since they do not keep track of flows. Their only possibility for a
guarantee is to guarantee the maximum rate of a whole class that has a separate
queue, however, this limits these IP packet routers to a small number of
guarantees. The market for ATM and Frame Relay has shown that real guarantees
can command much higher revenue. The method by which a flow router achieves a
guarantee is to compare the rate of the packet arrival in the state block to
the guaranteed rate and discard if the user exceeds the agreed rate by some
burst tolerance (just like ATM). The user can burst up to the limit of the
burst tolerance, but the rate is controlled on average. Since the rate is
known, the output port (and all in-between bottlenecks) can record the current
level of guaranteed traffic and reject (CAC) new guaranteed flows if the agreed
capacity is in use.

Guarantees for Flow Groups: Once a flow is
determined to be within its own rate limit, a flow router can check an
aggregate group of flows and discard the flow (UDP) or the packet (TCP) to
maintain a guarantee on the group of flows. This is important for VPN’s across
the network or for controlling the load on an external link feeding a standard
packet switching device (like an Ethernet switch) in front of the flow router.
This way the load through the external switch can be controlled so as to insure
all discarding and QoS decisions are made in the flow router, thus extending
the QoS functionality one or two stages beyond the flow router to the edge of
the network without replacing all the edge equipment.
Maximum Rate Traffic (UDP) CAC
Control: For UDP traffic where the rate is not
known, but the flows are also video or voice type calls where random packet
loss is not acceptable, the actual measured level of UDP traffic can be
measured and if a new UDP flow arrives when the measured level is at or above a
specified percentage of the trunk capacity then the new flow can be rejected by
discarding the first packet. The unique capability of a flow router is to know
which packets are ongoing flows (already in process) and which are new flows.
With variable rate compressed video the total UDP capacity for all the existing
flows will vary but if the UDP percentage is set correctly, the rejection of
new flows will keep the total UDP under the trunk capacity and thus prevent any
packet loss. This capability is one of
the strongest benefits of flow-based routing and is not available in
conventional IP routers.
TCP Slow Start Improvement: Today with random discard (RED or WRED) it takes
TCP longer than required to get up to the maximum rate that the network can
support. This is because random discards occur randomly as the flow is
increasing its rate, thereby causing TCP to drop back to half speed before
continuing to increase its rate. This substantially increases the time it takes
to transfer a web page. With flow routers, mp packets are discarded until each
TCP flow has reached the maximum rate the network can support and exceeded this
rate by the burst tolerance. Then packets will be discarded so that TCP will
cleanly oscillate around the rate the network has determined it can support for
all TCP flows in the same class. This should decrease the time to retrieve a
web page.

TCP Fairness and multiple SLA’s: In addition
to delaying slow-start, today’s IP routers using random discard allow different
flows on the same path to operate at different rates. This is because there is
no measurement of the flow rate and flows can obtain random rates due to the
random discards. Higher rates have more discards so there is some convergence
but still there is too much variation for a carrier to offer different service
levels (SLA’s). However, with flow routing, the flows can all be controlled to
the same rate and various weights can be applied so that one user could have
two or four times the bandwidth of another user who paid less (even if both are
TCP traffic types). This functionality is the same as ABR permitted in ATM and
this has been studied extensively in that literature (11).
High Trunk and Fabric
Utilization: As a result of being able to
dynamically balance the traffic load over multiple paths, if a flow router
measures the load on these paths or ports at frequent intervals (50 ms), it can
achieve over 80% utilization even with Internet type IP traffic, on all those
paths and ports. Thus, assuming sufficient TCP demand, a flow router network
could maintain a constant 80% average load on all the router’s ports, trunks
and internal fabric. This is possible due to the fact in those instances when
IP traffic can be dynamically distributed over many routes in very small
increments (individual flows) and controlled if the total is too high (using
TCP discard), the load on each path or port can be tightly held within about ±7%. Compare the
following:
A standard IP
packet router cannot dynamically balance any
load because anytime the routing of a destination is changed, all the flows can
get packets out of order. For UDP this is very bad, for TCP it costs
significant time. Thus, routing changes cannot be done at the fast (50 ms)
intervals required to achieve ±7% utilization. Any increase in the measurement
interval will have a linear impact on the utilization standard deviation. Thus
even if routing could be changed every ˝ second the utilization standard
deviation would be 70%, equivalent to no control at all. Thus, since route
changes without flow identification, are so harmful to the QoS due to packet
reordering, it is not feasible to achieve any gain in dynamic load balancing
with a best-efforts IP packet router. The gain that I have observed in studying
traffic and trunk utilization from many US Internet carriers is that there is
at least a 50% gain in trunk utilization that could be achieved through dynamic
load balancing (10).
A Flow Router however, can route each flow any way it wants to
without any out-of-order problem. Thus these devices can continuously change
their routing to optimize the network’s resources. This gain can be very significant and any such gain
correspondingly reduces the total network cost.
Mixing Packet and Flow IP Routers
Flow
routers and packet routers both interoperate totally since they both use the
save external protocols (IP or MPLS). Many of the cost improvements can be
achieved locally as flow routers are mixed into a network while still
preserving the investment in traditional router technology (and minimizing new
investments to strategic parts of the network). Also, many savings due to the
traffic control and SLA’s for new revenue opportunities can also be achieved
locally. However, in order to insure QoS improvements like bandwidth and delay
variation guarantees or the ability to CAC UDP flows, a flow router path across
the network must exist. The high QoS part of the network need not however, have
more capacity than is required by the premium real time streaming traffic, like
video and voice. Thus there are many mixed network possibilities where flow
routers only need to be added, as premium traffic demands their use.

Conclusion
IP
flow routers can significantly improve the QoS and utilization of an IP network
over the use on conventional IP packet routers. As a result, flow routers can
make possible the flawless support of many real-time streaming media like video
and voice using standard IP protocol.
In addition to managing the QoS of individual flows, they can manage
groups of flows so as to permit the use of inexpensive edge devices like
Ethernet switches for the final leg of an end-to-end QoS path through a
network. They also can permit many new service levels for higher revenue. And
lastly, they can permit a large cost reduction in the network through dynamic
load balancing and network convergence. Since they are 100% compatible with
today’s IP or MPLS packet routers, they can be mixed in a network, thereby not
requiring a full network replacement as voice and video support is introduced.
References
1. IP traffic and
QoS control: the need for a flow-aware architecture, T. Bonald, S.
Oueslati-Boulahia and J.W. Roberts, France Telecom R&D. Presented at World
Telecommunications Congress, September 2002 – Paris, France
2. Flow-aware
admission control for a commercially viable Internet, T. Bonald, S.
Oueslati-Boulahia and J.W. Roberts, France Telecom R&D. Presented at
EURESCOM Summit 2002, Heidelberg, Germany, October 2002
3. A flow-based
model for Internet backbone traffic, C. Barakat, P. Thiran, G.
Iannaccone, C. Diot, P. Owezarski. Proceeding of IMW 2002, ACM Press, Marseille
France, November 2002
4.The
performance of circuit switching in the Internet, Pablo
Molinero-Fernández, Nick McKeown, Stanford University, OSA Journal of Optical
Networking, Vol. 2, No. 4, pp. 83–96, March 2003
5.
T. Shimizu, M. Nabeshima, I. Yamasaki, T. Kurimoto "MXQ: A network
mechanism for
controlling misbehaving flows in best effort networks", IEICE Trans. Info
& Syst, Vol E84-D, Num. 5, May 2001
6.
I. Stoica, "Stateless Core: A scalable approach for QoS in the
Internet", Ph. D. Thesis, CMU, 2000.
7. Computer Network Development to
Achieve Resource Sharing – L. G. Roberts & B. Wessler,
Proceedings of the Spring Joint Computer Conference, Atlantic City, New Jersey
- May 1970
8. Evolution of Packet Switching
- Proceedings of the
IEEE, Vol. 66, No. 1 - November 1978
9. Toward
a Cooperative Network of Time-Shared Computers - with Thomas Marill) Proceeding of
the Fall Joint Computer Conference, San Francisco, California - November
1966
10. Internet Traffic Measurement
2000 and 2001, Lawrence G. Roberts, Jan 2001
11. Explicit Rate Flow Control, A 100 Fold Improvement over TCP, L. G. Roberts –
4/29/97
12. www.caspiannetworks.com