Home > Simulation: Simplify Network Management, testing and evaluation
- Whitepaper
Simulation: Simplify Network Management, Testing and Evaluation
- Whitepaper
Summary
Introduction
Challenges in Testing and Evaluation of Network Management Applications
Virtual Lab with Simulation Toolkit
Simulation Toolkit Applications
Development of the manager application
Testing the manager application
Fault Management testing
Performance Management testing
Scalability Testing
Testing Critical Network Scenarios
Evaluation of manager application
Product Demostration
Customer Support
Efficient Training
Simulation Toolkit Product Suite
Summary
With today's enterprise network infrastructure becoming more complex
and dynamic, a successful network management must ensure that the
network and applications are up-and-running smoothly and efficiently.
It must also possess the capability to predict problems before they
have any impact on the network. Recognizing the challenges of enterprise-grade
networks, an effective testing and evaluation of network management
system (NMS) becomes essential. This requires setting up of an extensive
laboratory with multi-vendor network devices, that are cost-prohibitive.
Moreover, networks are complicated and extremely expensive, and most
vendors cannot afford to set up a real network. To resolve these issues
WebNMS Simulation Toolkit a software suite that can simulate real
networks, was introduced. This paper explains how WebNMS Simulation
Toolkit can be used in the development, testing and evaluation of
network management applications.
Introduction
Network management is a service that employs a variety of applications
and devices to assist network managers in monitoring and maintaining
network integrity and performance. The Simple Network Management Protocol
(SNMP) is by far, the dominant protocol in network management. Every
SNMP-managed network consists of two key components: managed entities
and NMS applications. Every managed device (routers, servers, switches,
bridges, hubs, computer hosts, cable modems, printers, etc.) on the
network has an agent, which is a small program that communicates with
the Manager on one side and with Device or Application on the other
side The agent stores this information in a management information
base (MIB) and makes it available to the NMS by communicating through
the SNMP protocol. The NMS uses this information to monitor and control
the networked devices. Three versions of SNMP are currently prevalent:
SNMPv1, SNMPv2 and SNMPv3. All these versions conform to the specifications
of the structure of management information (SMI).
Network management generally involves device management/element
management, discovery management, fault management and performance
management.
Element Management Software (EMS) is specifically designed
to manage one or more of a specific type of telecommunications
network element (NE), for example a Router, Switch, Blade Server
etc.
Discovery management is the process by which the NMS discovers
devices and network components and provides details like the device
IPv4 or IPv6 Address, device name and device configuration.
Fault management functionality helps in reporting the errors
in the managed system. Alerts, such as server down, high servlet
execution time, etc. are reported and corrrective actions taken.
Performance management includes gathering statistical information,
determining system performance under natural and artificial conditions,
monitoring and collecting data from routers, switches, servers,
and any other devices, key information on errors, discards, and
critical performance factors such as memory and CPU utilization.
Challenges in Testing and Evaluation of Network Management Applications
As the number of manageable devices increases, it is important
to ensure that the network management system within a wide variety
of complex network environments. On an SNMP-managed network, each
managed device has an agent embedded and the device vendors frequently
release agent and MIB updates, which can cause network maintenance
problems. Network management developers and device vendors have
to ensure that their product is capable of managing and interacting
with all these devices. This requires extensive testing of their
products in real network environments. Using the production network
for testing purpose has its own limitations and risk. Setting up
a laboratory for this purpose is a another option but there are
certain difficulties associated with this. It requires procurement
of complex multi-vendor network devices. The time required to research,
budget, purchase and setup the lab requires human and financial
resorces that could be utilized more effectively.
Critical test scenarios, such as device failures and agent crashes
are difficult to reproduce with live devices. Hence, testing in
live environments is not a viable option. Moreover, software developers
may not have access to the actual devices, if the device is still
under development.
The ability of the NMS application to scale huge networks with
different type of devices is yet another challenge that needs
to be addressed. Scalability testing helps to determine whether
your management application scales with the workload growth
as the managed network grew in numbers and complexity. To achieve
the goals of scalability testing, a unique test bed for scalability
would need to be built. Inorder to assure that the NMS could
find the 1000s of devices typically installed across a customer’s
enterprise, engineers would need to spend massive amounts of
time and capital to build a hardware lab just for testing the
scalability of the new software.
A network comprises of various types of agents in the devices
and their different versions and protocols like SNMP, TFTP, FTP,
IOS, Telnet and TL1. IPv4 or IPv6 Address can be assigned to the devices. To configure these devices, experienced staff
with adequate knowledge in these protocols are required.
To validate the effective event handling of the manager application,
different type of traps and inform messages needs to be configured
and trap storms need to be generated at periodic intervals. In
a real network, events cannot be generated for testing purpose
as this would bring the entire network to a standstill.
Performance Testing involves testing for load, stress and measure
performance rates, such as discovery rate, trap processing rate,
and data collection rate of the management application. This involves
setting up of a large stable network that has the capability to
process such large amount of requests per second.
Cisco management application developers and QA groups need the
latest Cisco IOS hardware inorder to develop and test their software.
In the absence of the real device, they have to setup and maintain
a virtual lab populated with different types and versions of Cicso
IOS devices.
In general, the problems with lab maintenance and upgrades, the
financial burden involved and the varying requirements of the users
is a limitation for effective testing of the network management
application.
Virtual Lab with Simulation Toolkit
WebNMS Simulation Toolkit provides a solution to the above hurdles
in NMS testing and evaluation. Simulation Toolkit's Network Designer
offers an integrated network design studio to create a virtual
multi-vendor network environment with SNMP, TL1, TFTP, FTP,
Telnet and Cisco IOS manageable devices, on a single PC. The
device library consists of pre-configured devices that can be used
directly to create a virtual network populated with hosts, printers,
pix firewall, cisco series routers/switches and cable modem devices.
With Simulation Toolkit you can create a virtual network with 100,000+
network devices on a single PC. It enables testing and evaluation
of management applications for reliable functionality, scalability,
performance and disaster network scenarios.
Simulation Toolkit Applications
Development of Network Manager Application:
Network management application developers need to test their design
during each stage of their development with the required devices
and ensure that their application will adequately communicate with
and manage the network devices.
Simulation Toolkit simulates real devices by exporting MIB
object values but without the real device. The network management
application interacts with the simulated device just as it would
with real-world devices. To experiment with changes in the device
values, any variation on this simulation can be easily created by
configuring MIB values, simulation types and behavior scripts. This
enables developers to develop their applications in parallel with
agent development through simulation of device prototypes.
Simulation Toolkit enables simulation of thousands of agents on
a single PC, enabling simulation of large networks with different
device types. This results in a shortened development time, reduced
product development costs and an efficient network management software.
Testing Manager Application:
The manager application needs to be tested for fault management,
performance management, scalability and handling disaster network
scenarios. With Simulation Toolkit, testing can proceed faster and
high quality software products can be released to customers more
quickly.
1. Fault Management Testing
Fault management testing is required to validate the effective
event handling of the manager application. Simulation Toolkit's
Trap Simulation feature enables testing the Fault Management functionality
of the manager application.
Trap Configuration wizard enables configuration of SNMP
v1,v2,v3 traps with varbinds, that can be customized to be sent
at different scenarios on a daily or periodical basis, at the
specified time. For example, send link up/link down traps to update
the interface status, send cold start/warm start traps at agent
start up etc.
Trap Recording wizard can record SNMP v1 and v2c traps
sent by real device. The recorded traps can be stored in XML files
and replayed in the simulator as request-based, threshold-based
or time-based traps.
The Trap Stormer, an exclusive tool to storm SNMP v1/v2c
traps, can be used to validate the effective load handling of
the manager application for receiving any number of traps at the
specified time interval or at the burst mode.
2. Performance Testing
Performance Testing involves testing for load, stress and measure
performance rates, such as discovery rate, trap processing rate,
and data collection rate of the management application. Stress/Load
testing is subjecting the NMS to heavy data collection rates and
notification processing rates to validate the effective load handling
of the manager application for receiving any number of request/traps
at the specified time interval without any out-of-memory and other
related issues. This involves setting up of a large stable network
that has the capability to process such large amount of requests
per second.
Simulation Toolkit's Network Simulator enables you to
create a large network with multiple devices at the shortest possible
time. The Network Simulator had demonstrated support for 10,000+
device simulation on Windows platform and 100,000 device simulation
on a Linux platform, providing the opportunity to create network
environments that would be impractical to create in a physical
lab environment.
The Network Simulator is designed to provide enhanced network
performance for large networks. As per the Performance Metric
test taken for 50,000 nodes created for our customer test-bed,
the simulator was successfully started with 50,000 nodes and the
response time of 1804 ms was recorded when 250 nodes were queried
simultaneously by the NMS.
3. Scalability Testing
Scalability testing helps to determine whether your management
application scales with the workload growth as the managed network
grew in numbers and complexity. Hence, scalability testing forms
an essential part of the entire testing process of NMS. To achieve
the goals of scalability testing, a unique test bed for scalability
would need to be built to assure that the NMS could find the 1000s
of devices typically installed across a customer’s enterprise.
With Simulation Toolkit's Network Recorder, developers
and QA personnal can evenrecord real customer networks,
bringing that environment directly to the lab. The recorded network
can be replayed instantly in the Network Designer, thus enabling
the management applications to be tested in huge customer networks.
Simulation Toolkit's Integrated Network Designer can create
a large network with multiple devices, at the shortest possible
time. The Network Simulator had demonstrated support for upto
10,000 device simulation on a Windows platform and 100,000 device
simulation on a Linux platform, providing the opportunity to create
network environments that would be impractical to create in a
physical lab environment.
The Network Simulator provides the facility to add devices
in bulk at one shot with unique IPv4 or IPv6 Address and port number
combination. With option to add devices of the same type or different
type of devices at random, you can create the desired network
with ease. The virtual IPv4 or IPv6 Address is dynamically configured in
the system, when starting each agent instance in the network.
Simulation Toolkit's Topology Editor enables you to establish
inter-connections across routers, switches and other network devices
and visualize the topological relationship between the devices.
4. Testing Real Time and Critical Network Scenarios
Simulation Toolkit enables device configurations to be made from
UI or at runtime through scripts and RMI. The devices can also
be configured through FTP/TFTP and scripts can be executed on
file upload/download.
Simulation Toolkit's scripting capabilities enables real time
scenarios to be created easily, to replicate a real network.
For example, change device values and interface status, make addition/deletion
to SNMP table rows dynamically, start and shutdown devices, send
a trap to NMS on the start/stop of the device, add/delete devices
from network, configure and send traps in bulk for thousands of
agents etc. Device values can be made to vary or remain constant
based on the simulation types configured. Simulation Toolkit supports
Constant, Random, Linear, Wave, Exponential and Sine Wave simulation
types. The simulation scenarios can be enabled dynamically without
disrupting the simulation process.
Testing the manager application in critical network scenarios
helps to determine the managers' robustness in handling disaster
scenarios like network breakdown, delay in responses, agent crashes
etc.
Simulation Toolkit's scripting capabilities enable simulation
of critical network scenarios that are hard to reproduce in a
physical lab. Behavior scripts can be configued to simulate SNMP
v1, v2 error messages, runtime configurations like start/stop
of individual devices, up/down of device interface, delay in responses,
addition of new devices through scripts and RMI enable the management
software to be validated aganist large networks with real-time
crtitical scenarios.
Simulation Toolkit supports SNMP request and response PDU
customization. The SNMP PDU can be customized to send garbled/invalid
responses and test the manager application in handling such error
responses.
Simulation of real time and complex network scenarios enables
complete testing of network management applications. This enhances
product quality and results in a reliable and rugged management
applications.
Evaluating manager application:
Enterprises need to evaluate and finalize on the best management
application that suits their needs before purchasing the same. Testing
these products in real network environments will impact the real
network.
With Simulation Toolkit, users can easily record their real nework
and exhaustively evaluate new management products. The Jython script
APIs bundled with the product can be used to evaluate the capabilities
of the management products by creating "what if" scenarios,
which is impossible to reproduce in a real network.
Product Demonstration:
Demonstration of the management application at customer premises
or during trade shows is required to display the capabilities of
the product. Simulation Toolkit helps to create different network
scenarios instantly without the need for configuring and carrying
of devices.
The network configurations are portable across all OS. Any network
created in Simulation Toolkit can be packaged and installed in any
other system where Simulation Toolkit has been installed. This enables
marketing and sales to set up powerful "live" demonstrations of
their network management products at trade shows or at customer
premises, easily and effectively. Equipment Manufacturers can demonstrate
how their hardware scales and how its management application is
used. Reduced demo set up time and absence of bulky equipments results
in reduced cost of sales and easy and successful demonstrations.
Customer Support:
Problem identification in complex and huge customer network is
often time-consuming, complicated and involves huge expenses in
the support staff having to visit the customer network to reproduce
the problem.
With Simulation Toolkit , the support staff do not have to visit
the customer premises for problem reproduction. Its recording tool
can record the customer's network environment and reproduce the
problem quickly to provide fast and efficient support. Simulation
Toolkit's Network Recorder records real customer networks.
The recorded network can be saved and replayed instantly in the
Network Designer, thus enabling the management applications to be
tested in huge customer networks.
The ability to capture information on real networked devices, enables
developers and test engineers to support their customers with a
higher level of efficiancy.
Efficient Training:
With Simulation Toolkit, customized simulations can be created
and used to train network operators, administrators, and technicians.
Efficient training results in well trained and efficient network
management staff with better understanding of the network management
products and protocols.
Simulation Toolkit Product Suite
Simulation Tools:
Network Simulator
To design and simulate a network with SNMP, TFTP, FTP, Telnet
and IOS devices, on a single PC. Supports simulation of 100,000+
devices on a single PC. Jython-based scripting capabilities
enable modelling agent behavior and expressing inter-relationship
among MIB variables and across devices.
SNMP Agent Simulator
To simulate an SNMPv1/ v2c/v3 agent (or device) with user configured
data or recorded agent values. Supports Jython-based scripting
to model agent behavior and express inter-relationship among
MIB variables.
TL1 Agent Simulator
To smulate a TL1 Network Element with user configured data
. Supports Jython-based scripting to model agent behavior and
express inter-relationship among TCS commands.
SNMP Trap Stormer
To configure and send traps, to test the reliability of your
management application for receiving any number of traps at
the specified interval.
Network Recorder
To record real SNMP networks. The recorded network can be instantly
replayed in the network simulator.
Trap Recorder
To record real traps. The recorded traps can be replayed in
the trap configuration dialog of the SNMP Agent Simulator and
Network designer tools.
Manager Tools
MIB Browser: SNMP Manager application that enables loading,
browsing, walking the MIB tree, and performing all other SNMP-related
functions.
TL1 Craft Interface: TL1 manager application to test,
monitor, administer, and provision multiple TL1 agents
Editor Tools
Topology Editor: To establish inter-connections across
routers, switches and other network devices and visualize the
topological relationship between the devices.
SNMPv3 Configurator: To view and configure USM and VACM
user entries for the SNMP v3 user.
Telnet Command Editor: To view and edit supported telnet
commands.
Telnet Command Configurator: To add new telnet commands
to a specific device or to all the devices in the network.
IOS Command Editor: To view and edit supported Cisco
IOS commands.
IOS Command Configurator: To add new IOS commands to
a Cisco router/switch device in the network.
Device Library
Ready-to-use library of pre-configured devices bundled with
the product:
Host devices: Windows, Linux and Solaris
Printer devices: HP 4500 series
Router devices: Cisco 3600, Cisco 7600
Switch devices: Cisco 3750
Pix Firewall device
Cable Modem device
TL1 device
