The water potability dataset was retrieved from an open-source repository [30]. It contains a total of 3,276 records, with 1,998 records labelled as non-potable or unsafe for drinking and 1,278 records labelled as potable or safe for drinking. The dataset includes nine features: pH, hardness, solids, chloramines, sulfate, conductivity, organic carbon, trihalomethanes, and turbidity. Potability is represented by binary values, where 1 is for potable water and 0 is for non-potable water. These classifications are based on the concentration levels of the aforementioned substances and features.

The dataset was studied and investigated thoroughly to check its quality. It was found that there are 491 missing pH values, 781 missing sulfate values, and 162 missing trihalomethane values. The missing parameter values were replaced with the mean values, which is a common method, especially when the data are fairly symmetric.

The next step was to carry out exploratory data analysis (EDA). It includes examining histogram distribution for each feature, handling missing values (either by filling or removing), generating correlation analysis (heatmap), checking class balance for potability, and generating statistical summaries by calculating the mean, median, standard deviation, and minimum and maximum values. The results of this analysis were then presented visually. Figure 1 to Figure 4 show the distributions of pH, hardness, solids, and sulfate. From those figures, it can be seen that features are not perfectly normal, but many are skewed (e.g., solids, sulfate).

Figure 1.

Distribution of pH

Figure 2.

Distribution of hardness [mg/L]

Figure 3.

Distribution of solids [ppm]

Figure 4.

Distribution of sulfate [ppm]

Figure 5 shows the correlation heatmap of the nine features besides the potability. It can be seen that some positive correlations were observed (e.g., solids and conductivity). Most other features are weakly correlated with each other and with potability. In general, the weak correlation between features means the features are independent, and each feature has its own influence on the result of potability.

Figure 5.

Correlation heat map of the features

The dataset has class imbalance, with 1,998 records labelled as “Not Potable” and 1,278 records labelled “Potable”. This imbalance is an important consideration for designing machine learning algorithms later, as it may necessitate applying class-balancing techniques. To get a better idea about the features and their effect on potability, boxplots were generated to detect outliers, as shown in Figure 6 to Figure 9.

Figure 6.

Boxplot for pH by potability

Figure 7.

Boxplot for hardness [mg/L] by potability

Figure 8.

Boxplot for chloramines [ppm] by potability

Figure 9.

Boxplot for sulfate [ppm] by potability

From the above figures, it can be seen that for several features, the medians and distributions differ between potable and non-potable water, but often with overlap. Features such as pH, chloramines, sulfate, and trihalomethanes show visible differences.

Feature importance analysis was carried out using the Random Forest Gini Importance (also known as Mean Decrease in Impurity), which measures how each feature decreases impurity in classification trees [31]. It is a default and most commonly used technique in the scikit-learn library. The method calculates the total reduction in the impurity that each feature contributes across all decision trees in the Random Forest. Figure 10 shows the result of the feature importance analysis.

Figure 10.

Feature importance analysis result

The features above the importance median are pH, hardness, sulfate, chloramines, and solids. Accordingly, these five features are identified as the most relevant for predicting water potability. A refined version of the dataset was generated, which consists of the cleaned data (without any missing records) with the five features along with the potability label (0, 1).

One of the most commonly used metrics for classification algorithm performance assessment is accuracy, which, at the same time, is what is often reported [32]. However, it sometimes proves to be a deceptive and risky metric, especially with imbalanced datasets, where a class has more samples for one label than the other. In the dataset used in this work, there are 1,998 entries for the “Not Potable” class and 1,278 for the “Potable” class. The “Not Potable” class is therefore the majority class, whereas “Potable” is the minority class. If left unaddressed, machine learning algorithms tend to be biased towards predicting the majority class with deceptively high overall accuracy but poor performance when detecting the minority class [33]. Moreover, a model may have high accuracy because it is predicting the primary class; however, this does not necessarily indicate that the model has learned the true patterns within the dataset. Thus, it is vital to use other metrics such as the F1-score and AUC. The F1-score is the harmonic mean of precision (the proportion of true positive results among the total positive results) and recall (the proportion of total positives that were identified), which in turn indicates that the model is performing well in avoiding false positives and false negatives [32]. For example, while the model can accurately classify 80 safe samples out of 100 but inaccurately labels 40 unsafe samples as safe, the precision and recall will differ, and the F1-score will provide a balanced combined measure.

This approach is crucial when the cost of misclassification is high. The AUC evaluates the ability of the model to distinguish between classes for any given classification threshold. An AUC of 0.5 would indicate random guessing, while an AUC of 1.0 would indicate that the potable samples and non-potable samples are completely distinguishable. For instance, if the Random Forest algorithm has an AUC of 0.90, that means it will be accurate in 90% of the cases in ranking a randomly chosen potable sample over a non-potable sample. Accordingly, the F1 score and AUC will be considered (besides accuracy, precision, and recall) because they provide a more in-depth and reliable picture of the algorithm’s real-world performance than accuracy alone.

In this study, four classification models were selected for their unique features in binary classification problems with tabular real-world datasets such as the water potability dataset. RF, LR, XGBoost, and a DL model, which is the MLP, provide a balance between traditional machine learning tools, advanced ensemble models, and neural network-based approaches. Below is a more detailed justification, introduced with a summary shown in Table 1. Further, Table 2 provides the key parameters for each of the four models that were implemented using Python.

• RF Classifier: Random Forest was chosen for its robustness, interpretability, and performance with structured data. As an ensemble of decision trees, it reduces overfitting by averaging out predictions from many trees, which makes it very useful for complex nonlinear interactions in data. Also, its structure includes a feature importance analysis, which is a valuable tool for identifying the main water quality parameters.

• LR Classifier: Logistic Regression is a simple and interpretable baseline model. It is commonly used in the field of binary classification problems, and does very well when the relationship between input features and the target variable is almost linear. Also, it serves as a performance benchmark to which more complex models can be compared.

• XGBoost Classifier: Extreme Gradient Boosting was selected due to its outstanding performance and speed when processing structured datasets. Its features, such as regularisation and parallel computation, make it more advanced than typical boosting algorithms. As it is known for its ability to model nonlinearity and complex variable interactions, which makes it a top-performing algorithm in real-world applications, it is a key player in this study.

• Deep Learning MLP: A feed-forward neural network is a large-scale learning model that can learn complex high-level abstractions from the data. Though it does not always perform best on small to medium-sized tabular datasets, Deep Learning can outperform other models when scaled up in terms of data input and tuning. It also serves as a tool to see how well a neural architecture generalises, which is in contrast to tree-based models and linear classifiers.

As noted, the dataset has a class imbalance – an issue that must be addressed. One of the commonly used methods in this regard is SMOTE (Synthetic Minority Over-sampling Technique) [34], [35]. It generates artificial samples of the minority class by interpolating existing samples in feature space to allow the classifier to learn the decision boundary in a better way. SMOTE was applied to the training set only to avoid the data leakage problem.

The hyperparameter tuning method is used to find the best set of parameters for ML models without modifying the algorithm itself. It occurs before training. It optimises model performance in such a way that it enhances accuracy and generalisation (performance on new data), reduces overfitting or underfitting, and increases training efficiency (speed, memory usage).

Hyperparameter tuning was also conducted for the four models using GridSearchCV and 5-fold stratified cross-validation. The method tried various combinations of parameters to identify the optimal configuration according to the F1-score, improving the generalisation ability and precision of the used models.

To assess the external validity of the proposed approach, the findings of this work will be compared with the results in existing literature. For example, the use of ensemble models (RF and XGBoost) will be compared with the performance of these models shown in references such as [10], [18], [24], while the performance of SMOTE in imbalanced data will be assessed by studies such as [34], [35]. Although the dataset differs from other works in geography or water source, performance trends can provide a good understanding and validation for the methodological choices of this research.

After assessing the performance of each model, the best model will be exported (using joblib) and integrated into a user-friendly GUI application to be built with Streamlit. The application allows users to enter water characteristics and receive real-time predictions on potability (i.e., safe or unsafe to drink). More details will be provided in the next section.

The first experiment, conducted on the refined dataset after handling missing values and selecting only the five top features, is to generate a performance comparison of the four models, namely, RF, LR, XGBoost, and DL MLP. It should be noted that the dataset used in this experiment still suffers from class imbalance.

After running the four models, performance metrics were calculated based on the obtained results. Table 3 shows a summary of the metric values for the four models.

Table 2 above highlights the superior performance of RF and XGBoost, where they both outperform LR and DL in terms of F1-score and AUC. The poor performance of the LR and the DL neural network model is likely due to class imbalance and a lack of model complexity or tuning. Figure 11 shows the F1-score results, while Figure 12 shows the AUC score results of the four models.

Figure 11.

F1-Score Results of the Four Models

Figure 12.

AUC score results of the four models

From Figure 11, XGBoost slightly outperformed the RF model, while LR and DL have shown poor F1 scores due to class imbalance or underfitting problems. From Figure 12, it can be seen that the RF model has shown the best AUC (which represents the discriminative power), followed closely by XGBoost. LR and DL performed almost randomly (since AUC ≈ 0.5).

Figure 13 highlights the ROC curve comparison for the four tested models, which confirms the findings stated above about the four models. Based on the metrics and considering the importance of correctly classifying potable and non-potable water, Random Forest can be considered the best-performing model for this task, closely followed by XGBoost. Random Forest has the highest AUC (0.684) and a decent F1-score (0.468), while XGBoost has a slightly lower AUC (0.664) but a slightly better F1-score (0.485). Random Forest achieved the highest AUC score, indicating better overall class discrimination ability, and XGBoost achieved the highest F1-score, reflecting a slightly better balance between precision and recall, especially when both classes are important. Logistic Regression and Deep Learning failed completely (precision, recall, and F1-score are all zeros). This likely means the models are predicting only one class (probably “non-potable”) and not learning properly. The models still have room for improvement because the best F1-scores are below 50%. This result suggests either that the features were not fully predictive or that models need better tuning (hyperparameter optimisation) or that more complex features (e.g., interaction terms) are needed.

Figure 13.

ROC curve comparison for the four models

Data imbalance is a known issue in classification applications that can lead to biased models favouring the majority class while hindering performance on the minority class. Among others, SMOTE is used to address this problem and increase the number of samples in the minority class. It is a popular technique for handling class imbalance by creating synthetic examples of the minority class (in this case, “potable” water) rather than simply duplicating them.

In this work, hyperparameter tuning was implemented using GridSearchCV, which is part of the scikit-learn Python library. It is a brute-force search technique that evaluates all possible combinations of specified hyperparameters using cross-validation.

Feature engineering is the process of transforming raw data into useful features for machine learning models. These engineered features capture complex interactions that may not be obvious in the default datasets. Some of the features can be selected, and new features can be created through the transformation of features. Such a process can optimise the model’s performance and make it more efficient and accurate. Four new features, using operations such as ratios, products, and differences, were added, and the performances of the four models were analysed again by incorporating the new features in the dataset. The additional new features are provided in Table 4.

After carrying out the above improvements, the models were trained and tested again. Table 5 shows the performance metrics results for the four models. It can be seen from the table that there are noticeable improvements as a result of the SMOTE-based class balancing, hyperparameter tuning, and feature engineering (e.g., hardness/solids ratio, sulfate interactions). The F1-score and AUC for all the models have improved considerably.

From the above table, it can be seen that XGBoost and Random Forest models have achieved a more balanced classification between the “safe” and “unsafe” water classes, with much improved F1-score and AUC. Logistic Regression and Deep Learning models have also improved significantly after balancing the dataset and feature engineering, but showed slightly lower recall and F1-score.

Figure 14 shows the F1-score results of the four models after the above adjustments and improvements, while Figure 15 shows the AUC score results of the four models after the same adjustments and improvements.

Figure 14.

F1-Score Results of the Four Models after Improvements

Figure 15.

AUC Score Results of the Four Models after Improvements

XGBoost has shown the highest F1-score (0.86), followed by Random Forest (0.85). Logistic Regression and Deep Learning performed slightly lower. XGBoost also has the highest AUC (0.91), which confirms its superior ability to distinguish between safe and unsafe water, and Random Forest follows closely with 0.90. Logistic Regression and Deep Learning are slightly behind. Accordingly, the XGBoost model offers the best overall balance between precision and recall. Deep Learning achieved decent performance but did not outperform ensemble methods on this tabular dataset.

In summary, XGBoost is the top-performing model with the highest AUC (0.91) and an excellent F1-score (0.86). Random Forest closely follows with an AUC of 0.90 and strong classification metrics. Logistic Regression provided a strong baseline with good performance after balancing and tuning. The Deep Learning MLP was effective but less optimal than ensemble models in this context. As a result, the XGBoost model was selected for deployment due to its strong performance, interpretability, balance of accuracy, and robustness.

Lastly, an application was created using Python joblib and Streamlit. The application provides the user with a prediction of whether the water is drinking “SAFE” or “UNSAFE” after entering the values of the required chemical properties or the feature values on the GUI. The XGBoost model was saved (exported) using the joblib library. Then, the application was built using Streamlit, which is a Python-based framework for rapid app deployment. Figure 16 shows two examples of executing the application. The app can be implemented on mobile devices, PCs, or other platforms, allowing easy visual interaction with users interested in predicting water potability.

Figure 16.

Water Potability Prediction App – safe/unsafe water prediction outputs

To ensure the external validity of this work's findings, the performances of the used models are compared to results in existing water quality prediction research. Previous relevant studies have shown that more advanced ensemble methods, such as Random Forest and gradient boosting algorithms, e.g., XGBoost, outperform baseline linear models or logistic regression. Apart from this, class balancing methods such as SMOTE have been widely verified in environmental as well as general machine learning applications to be efficient tools against imbalanced datasets. Table 6 summarises this work's findings versus those available in recent literature. Evidence presented in Table 6 verifies that the obtained results are aligned with previous studies. Specifically, ensemble models always perform well on different datasets and geographic locations, which indicates the robustness and generalisability of the used models. Also, the use of SMOTE and hyperparameter tuning conforms to the state-of-the-art practice in the literature and further justifies their necessity in improving model fairness and prediction performance in imbalanced datasets. While the dataset used in this work differs in size and origin from the ones described in the literature, the uniform patterns of performance across studies confirm the accuracy and stability of the proposed approach.

While this study demonstrates the applicability of ensemble and boosting techniques, class balancing, and feature engineering in water potability prediction, a few limitations exist. First, the study is limited to one publicly available dataset, which constrains the geographic and temporal range of water quality parameters under consideration. Therefore, the findings may not necessarily apply fully to water sources with other characteristics. Second, the validation method is limited to held-out test data from the same dataset. Although stratified train–test splits and cross-validation minimise overfitting, more comprehensive blind testing across external datasets from other sources or under different environmental conditions is necessary for external validation. Third, interpretability is addressed through feature importance analysis on ensemble models, but advanced interpretability tools (e.g., SHapley Additive exPlanations) were not implemented in this work. These may provide greater model decision-making insight, especially in neural networks. Finally, real-world deployment concerns such as sensor fusion, data streaming, and hardware efficiency are beyond the scope of this work, although the prototype Streamlit application demonstrates practical feasibility. These limitations are areas for future investigations. Generalisation to multiple datasets, using advanced explainability techniques, and testing in realistic operating environments would enhance the validity and generalizability of the proposed models.

To show the impact of the improvement after class balancing, hyperparameter tuning, and feature engineering, one can compare the results shown in Table 3 and Table 5. As can be seen from this comparison, all four models improved their performance significantly, with Random Forest and XGBoost improving the most. For example, Random Forest F1-score improved from 0.47 to 0.85, and XGBoost from 0.49 to 0.86, while Logistic Regression and Deep Learning MLP also saw a significant improvement from near-random performance to competitive performance.

It should be noted, though, that after feature engineering, the differences between the two top models (Random Forest and XGBoost) are very small, with XGBoost being only ~1% superior in F1-score and AUC. This result means that feature engineering is the process that gets all models to a strong and comparable performance. The marginal gain of XGBoost may justify its selection as the deployment model, but Random Forest is a close second. RF may be preferred in circumstances where simpler interpretability or reduced computational cost is desirable.