Test Problem Suite

Test Problems and Test Data for Multiobjective Optimizers

Authors: Eckart Zitzler, Marco Laumanns

This page contains test data sets and experimental results to some of our papers on evolutionary multiobjective optimization. For each test problem, the following information is available:

• The paper where the test problem is defined and used.
• If necessary, test data sets for the corresponding problem.
• Results achieved by different evolutionary multiobjective optimizers considered in the corresponding paper.

The test data sets are available in ASCII format, the format is self-explaining. The experimental results regarding the evolutionary algorithms can be downloaded as gzip'ed tar files, where each tar file contains for each algorithm and optimization run a separate ASCII file (the format is explained in the next section). The shortcuts of the algorithms correspond to the ones used in the papers.

File Format

The outcome of each optimization run is stored separately in an ASCII file for each algorithm under consideration. The file name consists of the shortcut for the algorithm and the number of the optimization run. For instance, SPEA.1 refers to the strenght Pareto evolutionary algorithm [ZT1999a] and contains the outcome of the first optimization run.

Each file consists of a list of points in the objective space, which represent the nondominated front achieved by the corresponding algorithm in the corresponding run. Each line in the file describes one point of the trade-off front and contains a sequence of numbers, separated by blanks. Per line, the first number corresponds to the first objective, the second number to the second objective, and so forth. The objectives are numbered as in the papers, i.e., for the knapsack problem the first objective is the profit of the first knapsack, with the test functions f1 is the first objective.

Test Problems

Multiobjective 0/1 Knapsack Problem

References

• [ZLT2001a], [ZT1999a], [ZT1998b], [ZT1998a] (the experimental results that can be downloaded here are the ones presented in [ZLT2001a] and [ZT1999a])

Test Data:

• 100 items, 2 knapsacks
• 250 items, 2 knapsacks
• 500 items, 2 knapsacks
• 750 items, 2 knapsacks
• 100 items, 3 knapsacks
• 250 items, 3 knapsacks
• 500 items, 3 knapsacks
• 750 items, 3 knapsacks
• 100 items, 4 knapsacks
• 250 items, 4 knapsacks
• 500 items, 4 knapsacks
• 750 items, 4 knapsacks

Pareto-optimal fronts (all Pareto-optimal objective vectors):

• 100 items, 2 knapsacks
• 250 items, 2 knapsacks
• 500 items, 2 knapsacks
• 100 items, 3 knapsacks

Experimental Results:

[ZT1999a]:

• 100 items, 2 knapsacks
• 250 items, 2 knapsacks
• 500 items, 2 knapsacks
• 750 items, 2 knapsacks
• 100 items, 3 knapsacks
• 250 items, 3 knapsacks
• 500 items, 3 knapsacks
• 750 items, 3 knapsacks
• 100 items, 4 knapsacks
• 250 items, 4 knapsacks
• 500 items, 4 knapsacks
• 750 items, 4 knapsacks

[ZLT2001a]:

• 750 items, 2 knapsacks, 125000 fitness evaluations
• 750 items, 2 knapsacks, 240000 fitness evaluations
• 750 items, 2 knapsacks, 480000 fitness evaluations
• 750 items, 3 knapsacks, 150000 fitness evaluations
• 750 items, 3 knapsacks, 288000 fitness evaluations
• 750 items, 3 knapsacks, 576000 fitness evaluations
• 750 items, 4 knapsacks, 175000 fitness evaluations
• 750 items, 4 knapsacks, 336000 fitness evaluations
• 750 items, 4 knapsacks, 672000 fitness evaluations

Continuous Test Functions

References:

• [ZLT2001a], [ZDT2000a], [Zitz1999], [ZDT1999a], [ZDT1999b], [Deb1998a] (the experimental results that can be downloaded here are the ones presented in [ZLT2001a] and [ZDT2000a])

Experimental Results:

[ZDT2000a]:

• test function 1
• test function 2
• test function 3
• test function 4
• test function 5
• test function 6

[ZLT2001a]:

• SPH-2, 2 objectives, 100000 fitness evaluations
• SPH-2, 2 objectives, 500000 fitness evaluations
• SPH-2, 2 objectives, 1000000 fitness evaluations
• SPH-3, 3 objectives, 100000 fitness evaluations
• SPH-3, 3 objectives, 500000 fitness evaluations
• SPH-3, 3 objectives, 1000000 fitness evaluations
• QV, 2 objectives, 100000 fitness evaluations
• QV, 2 objectives, 500000 fitness evaluations
• QV, 2 objectives, 1000000 fitness evaluations
• KUR, 2 objectives, 100000 fitness evaluations
• KUR, 2 objectives, 500000 fitness evaluations
• KUR, 2 objectives, 1000000 fitness evaluations
• ZDT6, 2 objectives, 100000 fitness evaluations
• ZDT6, 2 objectives, 500000 fitness evaluations
• ZDT6, 2 objectives, 1000000 fitness evaluations

Source Code

Please also visit the PISA Website for a collection of MOEA implementations.

• C code for the calculation of the size of the dominated space (hypervolume metric)
• SPEA - Strength Pareto Evolutionary Algorithm for multiobjective optimization (C++ program code)
• SPEA implementation for the Traveling Salesperson Problem on the basis of the FEMO library (C++ code)

References

E. Zitzler, M. Laumanns, and L. Thiele. SPEA2: Improving the Strength Pareto Evolutionary Algorithm. Technical Report 103, Computer Engineering and Communication Networks Lab (TIK), Swiss Federal Institute of Technology (ETH) Zurich, Gloriastrasse 35, CH-8092 Zurich, May 2001.

[ZDT2000a]

E. Zitzler, K. Deb, and L. Thiele. Comparison of Multiobjective Evolutionary Algorithms: Empirical Results. Evolutionary Computation, 8(2), pages 173-195, Summer 2000.

[Zitz1999]

E. Zitzler. Evolutionary Algorithms for Multiobjective Optimization: Methods and Applications. PhD Thesis, Swiss Federal Institute of Technology (ETH) Zurich, November 1999. Supervisors: Lothar Thiele, Kalyanmoy Deb.

[ZT1999a]

E. Zitzler and L. Thiele. Multiobjective Evolutionary Algorithms: A Comparative Case Study and the Strength Pareto Approach. IEEE Transactions on Evolutionary Computation, 3(4), pages 257-271, November 1999.

[ZDT1999b]

E. Zitzler, K. Deb, and L. Thiele. Comparison of Multiobjective Evolutionary Algorithms on Test Functions of Different Difficulty. Genetic and Evolutionary Computation Conference (GECCO-99): Bird-of-a-feather Workshop on Multi-criterion Optimization Using Evolutionary Methods, July 1999.

[ZDT1999a]

E. Zitzler, K. Deb, and L. Thiele. Comparison of Multiobjective Evolutionary Algorithms: Empirical Results. Technical Report 70, Computer Engineering and Communication Networks Lab (TIK), Swiss Federal Institute of Technology (ETH) Zurich, Gloriastrasse 35, CH-8092 Zurich, February 1999.

[Deb1998a]

K. Deb. Multi-objective genetic algorithms: Problem difficulties and construction of test functions. Technical Report No. CI-49/98, Department of Computer Science/XI, University of Dortmund, Germany, 1998.

[ZT1998b]

E. Zitzler and L. Thiele. Multiobjective Optimization Using Evolutionary Algorithms - A Comparative Case Study. Parallel Problem Solving from Nature - PPSN-V, pages 292-301, Springer, Berlin, September 1998.

[ZT1998a]

E. Zitzler and L. Thiele. An Evolutionary Algorithm for Multiobjective Optimization: The Strength Pareto Approach. Technical Report 43, Computer Engineering and Communication Networks Lab (TIK), Swiss Federal Institute of Technology (ETH) Zurich, Gloriastrasse 35, CH-8092 Zurich, May 1998.