Benchmarking Methodology (bmwg)

Charter

Chair(s):
     Scott Bradner, sob@harvard.edu

Mailing Lists:
     General Discussion:  bmwg@harvisr.harvard.edu
     To Subscribe:  bmwg-request@harvisr.harvard.edu
     Archive:

Description of Working Group:

     The major goal of the Benchmark Methodology Working Group is to
     make a series of recommendations concerning the measurement of
     the performance characteristics of different classes of network
     equipment and software services.

     Each recommendation will describe the class of equipment or
     service, discuss the performance characteristics that are
     pertinent to that class, specify a suite of performance
     benchmarks that test the described characteristics, as well as
     specify the requirements for common reporting of benchmark
     results.

     Classes of network equipment can be broken down into two broad
     categories.  The first deals with stand-alone network devices
     such as routers, bridges, repeaters, and LAN wiring
     concentrators.  The second category includes host dependent
     equipment and services, such as network interfaces or TCP/IP
     implementations.

     Once benchmarking methodologies for stand-alone devices have
     matured sufficiently, the group plans to focus on methodologies
     for testing system-wide performance, including issues such as
     the responsiveness of routing algorithms to topology changes.


Goals and Milestones:


Done       Issue a document that provides a common set of definitions
           for performance criteria, such as latency and throughput.

Done       The document will also define various classes of stand-alone
           network devices such as repeaters, bridges, routers, and LAN
           wiring concentrators as well as detail the relative
           importance of various performance criteria within each class.

                                   1





TBD        Once the community has had time to comment on the definitions
           of devices and performance criteria, a second document will
           be issued.  This document will make specific recommendations
           regarding the suite of benchmark performance tests for each
           of the defined classes of network devices.



                                   2