Unit 6: Data and Analysis — Short Questions

Data science is the process of collecting, organising, analysing, and interpreting data to find useful information and support decision-making. It is an interdisciplinary field that combines mathematics, statistics, computer science, domain knowledge, and communication skills to convert raw data into meaningful knowledge.

The two main types of data are structured data, which is organised in a fixed format such as tables with rows and columns (e.g., databases, spreadsheets), and unstructured data, which does not follow a fixed format (e.g., images, videos, emails, and audio files).

Structured data is organised in a clear format with rows and columns, such as a student mark sheet or database, and is easy to understand and analyse. Unstructured data does not follow a fixed format, such as emails, images, audio, and videos, and is difficult to analyse directly, requiring special tools to process it.

Two common data collection methods are surveys and sensors. (Other valid examples given in the chapter are interviews, forms, and online platforms such as blogs and social websites.) Data can be collected manually, using human effort, or automatically, using machines and software.

Machine learning is a subfield of data science that enables systems to improve from experience without explicit programming. It analyses patterns in data using large datasets to train models, so that the system can make predictions or decisions and improve over time as new data is introduced.

Supervised learning is a machine learning method in which the model is trained using labelled data, where each data item has an input and a correct output. The system learns by comparing its output with the correct answer, and it is used for prediction tasks such as email spam detection and exam result prediction.

The purpose of feature selection is to choose the input values (features) that are most relevant to the problem so that the model can understand the factors affecting the outcome. Correct feature selection improves learning and model performance, whereas unnecessary or irrelevant features can reduce accuracy.

Accuracy measures how many predictions made by the model are correct out of the total predictions, expressed using Accuracy = (TP + TN) / (TP + TN + FP + FN). High accuracy means better overall performance, while the error rate shows the proportion of incorrect predictions.

Prediction estimates what may happen in the future based on patterns found in historical data (forecasting outcomes) without necessarily explaining why it happens. Causality explains why an event happens by identifying the cause-and-effect relationship between variables, showing how one factor directly influences another.

Python (used with libraries such as Pandas and NumPy, often through Jupyter notebooks) is one commonly used tool for machine learning. (Excel and R are other acceptable answers.)