4.2 Introduction to Using Data Sets

What you need to know

• A data set is a collection of related data, often organized as records (one per item) with fields (attributes of each item) — like a spreadsheet's rows and columns.
• In Java, a data set is typically stored as an array or ArrayList of values, or of objects where each object is one record.
• Data may come from user input, a text file (4.6), or be built in code.
• Before analysis, data often needs cleaning: removing invalid or duplicate entries, standardizing formats, handling missing values.
• Common analyses: summary statistics (sum, average, max, min), filtering records that meet a condition, counting matches, sorting, and finding patterns.
• The structure of the data determines which collection to use: fixed-size known-length → array; growing/shrinking → ArrayList; grid → 2D array.

Worked example

// a data set of daily temperatures as an array
double[] temps = {72.5, 68.0, 75.2, 71.8, 69.9};
double sum = 0;
for (double t : temps)
{
    sum += t;
}
double avg = sum / temps.length;   // 71.48

// a data set of records as an ArrayList of objects
ArrayList<Student> roster = new ArrayList<Student>();
roster.add(new Student("Ava", 92));
roster.add(new Student("Ben", 85));

Going deeper

The nuance, edge cases, and connections that turn a 3 into a 5.

• A data set is structured: records (rows) with fields (columns). In Java, that's usually an array or ArrayList where each element is one record — either a primitive (one field per record) or an object (many fields per record).
• Choosing the structure: known fixed size → array. Size changes → ArrayList. Grid/table with rows and columns → 2D array. Records with multiple fields → array or ArrayList of objects of a class you define.
• Cleaning before analysis: consistent format (dates, capitalization, units), handle missing values (skip, default, or flag), remove duplicates, validate ranges. Without cleaning, an average with a stray −999 sentinel value is garbage.
• Transformation: computing new fields from existing ones — a BMI from height and weight, a total from unit price and quantity. Usually a loop that reads fields and writes a new value.
• Summary statistics (count, sum, mean, min, max, frequency) are the sum/count/max algorithms from Unit 2 applied to a data set. Filtering (records matching a condition) is a loop with an if. Sorting (4.15) enables binary search and ordered output.
• Data comes from user input (Scanner), files (4.6), or is hard-coded for testing. The processing code is the same regardless of source.
• The revised CED added this topic so that FRQ 3 (ArrayList data analysis) has a conceptual foundation: you're not just manipulating a list, you're analyzing a data set.

Mistakes that cost points

• Choosing an array when size is unknown. ArrayList.
• Analyzing before cleaning. Questions that show inconsistent data want cleaning first.
• Storing records as parallel arrays. An array of names and a separate array of scores is fragile. One object per record is the CED's model.

Practice questions

Written in the style of the real exam. Try each one before revealing the answer.