We want to study a new outlook for the implementation of a network monitoring infrastructure, that converges towards a Service Oriented view of the Internet.
When applied to the specific case of network monitoring, such perspective is characterized by the following features:
monitoring activity is controlled by monitoring requests that come from the outside of a cloud that offers services that involve communication;
monitoring activity is managed accoding to the internal structure of the cloud which is opaque to the clients.
Such view is distant from the perspective used in the design of most network monitoring infrastructures, that are traditionally oriented to network administration: in our case the target user is not the network administrator that wants to diagnose a problem or to optimize a network, but rather a client application that wants to keep under control the unfolding of a distributed processing taking place inside the cloud.
In order to justify our approach, we introduce two case studies. The first one addresses a service provider that mainly provides connectivicty, and is therefore telecom oriented. The second considers a provider with a richer offer, supported by connectivity.
A company U wants to use the connectivity offered by a provider P to support an internal p2p application (for instance, an internal telephony service of geographical extent). Company U wants to monitor the effective use of the service, in order to optimize the service level agreement with the provider P. To this purpose the company U requires that the bandwidth effectively used is reported to its administrative office, ensuring privacy.
We note that, upon issuing the request, the client U is unable to indicate the relevant parameters for the configuration of the network monitoring activity: the address of the points of presence that provide the service, that of the routers that support the communication, and possibly also the IPs of the final destinations are unknown. These pieces of information are locked inside the cloud that provides the service, and the service provider P is not willing to make them accessible from outside.
However, the provider may offer network monitoring data collection as a complementary service, that integrates the basic connectivity. It will be task of the client to specify what kind of data to collect, how frequently, etc., and we expect that the client will also take care of storing and processing network monitoring data.
We note that this case regards a quite static configuration of the resources needed to perform network monitoring inside the cloud. The next use case introduces a further grade of dinamism.
A provider offers complex computational services, in Grid Computing style. Some require a relevant communication load, for instance to move data from storage to a processing element. The user wants to evaluate the need of using an expensive option that secures a given bandwidth in data transfers.
As in the above case, the user is unable to address the involved resources: the cloud offers data processing services that are later allocated to available resources. The scheduler may accomodate specific requests from the client (some sort of co-location in our case), but the implementation of the service remains opaque for the client.
Unlike the previous case, here the dynamic activation of a short lived network monitoring session is required: after jobs are allocated to the appropriate resources, the network monitoring infrastructure inside the cloud will manage to activate the monitoring of the appropriate network elements.
The topics opened by a Service Oriented approach to network monitoring address two cornerstones of distributed computing: security and scalability. We observe that a Service Oriented approach offers a way to define the challenges introduced by these two characteristics. And a good definition is one step towards a solution.
Security addresses the relationship between a client and a service provider: the borderline through which communications need to be authenticated is defined in an unambiguous way. Although this is not the solution of the problem, however it is a clear indication about how to proceed: the service provider will need to maintain a database for client authentication, and associate (communication) resources to clients. Inside the cloud, data and control information will not need to be authenticated, although encryption is appropriate for privacy reasons.
The comparison with an unstructured approach is favorable: in a sysadmin oriented network monitoring infrastructure, agents collect every data that is possibly of interest at a later point in time, and make it available with a hardly predictable granularity to a number of restricted administrators groups. The problem is poorly defined, and hard to approach.
In a Service Oriented approach to Network Monitoring the challenges introduced by security are well defined: client authentication, resource management inside the cloud, and secure transfer of data from the cloud to the client.
Scalability issues concern in distinct ways the number of clients, and the internal structure of the cloud. An effective scalability with respect to the number of clients is a problem of a Service Oriented Architecture as a whole: Network Monitoring is not in charge of addressing such aspect. Instead, the internal structure of the cloud may make a problem, when traffic is not structured and metered in a selective way. In a Service Oriented view, clients provide the patterns to select what traffic is to be metered, and how.
Also in this case, the comparison with an unstructured approach is favorable: in a sysadmin oriented infrastructure, agents collect data according to a sysadmin view of the network. Since data tend to explode with the complexity of the network, some aggregation criteria must be applied. But aggregation criteria that are useful for the sysadmin (e.g, TCP traffic from router A to router B) are not such for the application: indeed, the application client is the good candidate to provide such information.
In a Service Oriented approach scalability is effectively addressed by the fact that Network Monitoring activity is controlled by the clients, that give explicit indications about what traffic is to be monitored, and how to aggregate records. However, monitoring tools must support a dynamic configuration, and a way to instruct the cloud about data aggregation must exist.
The project we have in mind starts from the achievements of the FP6 - Network Monitoring research group inside the Institute for Resource and Workflow Monitoring. Such project started from generic statements of scalability and security, but we did not feel the need as overlay structure to define the nature of a network monitoring request. During the unfolding of the project, the existence of a latent Service Oriented view became evident. The prototype implemented at the end of the project confirms such outlook.
Our intention is to proceed in a similar way: investigate the abstract issues related with our approach and proceed towards a deployable prototype. Building blocks should be mainly found in existing stable open source software: only in case certain peculiar funtionalities are missing, we may undertake the task of designing ad-hoc software components.We expect from this project a scientific advancement in the design of network monitoring infrastructure through the use of a Service Oriented paradigm, and a better knowledge of existing tools and concepts for Service Oriented Architectures.