Abstract
This note introduces a new attribute selection measure for ID3-like inductive algorithms. This measure is based on a distance between partitions such that the selected attribute in a node induces the partition which is closest to the correct partition of the subset of training examples corresponding to this node. The relationship of this measure with Quinlan's information gain is also established. It is also formally proved that our distance is not biased towards attributes with large numbers of values. Experimental studies with this distance confirm previously reported results showing that the predictive accuracy of induced decision trees is not sensitive to the goodness of the attribute selection measure. However, this distance produces smaller trees than the gain ratio measure of Quinlan, especially in the case of data whose attributes have significantly different numbers of values.
Article PDF
Similar content being viewed by others
References
Bratko, I., & Kononenko, I.(1986).Learning diagnostic rules from incomplete and noisy data.Seminar on AI Methods in Statistics.London.
Breiman, I., Friedman, J., Olshen, R., & Stone, C.(1984).Classification and regressing trees.Belmont, CA: Wadsworth International Group.
Cestnik, B., Kononenko, I., & Bratko, I.(1987).ASSISTANT 86:A knowledge-elicitation tool for sophisticated users.In I. Bratko & N. Lavrac (Ed.), Progress in machine learning,Sigma Press.
Clark, P., & Niblett, T.(1987).Induction in noisy domains.In I. Bratko, & N. Lavrac (Eds.), Progress in machine learning,Sigma Press.
Hart, A.(1984).Experience in the use of an inductive system in knowledge engineering.In M. Bramer (Ed.), Research and developments in expert systems.Cambridge University Press.
Kononenko, I., Bratko, I., & Roskar, E.(1984).Experiments in automatic learning of medical diagnostic rules.(Technical Report)Ljubljana, Yugoslavia: Jozef Stefan Institute.
Lopez de Mantaras, R.(1977).Autoapprentissage d 'une partition:Application au classement iteratif de donnees multidimensionelles.Ph.D.thesis.Paul Sabatier University, Toulouse (France).
Mingers, J.(1989).An empirical comparison of selection measures for decision-tree induction.Machine learn-ing, 3, 319-342.
Quinlan, J.R.(1979).Discovering rules by induction from large collections of examples.In D. Michie (Ed.), Expert systems in the microelectronic age.Edinburg University Press.
Quinlan, J.R.(1986).Induction of decision trees.Machine learning, 1, 81-106.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
De Mántaras, R.L. A Distance-Based Attribute Selection Measure for Decision Tree Induction. Machine Learning 6, 81–92 (1991). https://doi.org/10.1023/A:1022694001379
Issue date:
DOI: https://doi.org/10.1023/A:1022694001379