Makieu, Peter2026-10-192025-10-23Makieu, P., Mutwiri, J., & Martor, J. J. (2025). Evaluating the Efficiency and Performance of Minimalist Neural Network Architectures with Hybrid Activation Functions. Journal of Advance Research in Computer Science & Engineering, 10(2), 12–27. https://doi.org/10.61841/5zs7sn3910.61841/5zs7sn39https://doi.org/10.61841/5zs7sn39https://dspace.njala.edu.sl/handle/20.500.144402/469This research introduces an Adaptive Hybrid Activation Function (AHAF) for minimalist neural networks, addressing efficiency and performance challenges in resource-constrained applications.Unlike existing hybrid activations (e.g., Swish) or minimalist approaches (e.g., Lottery Ticket Hypothesis (LTH)), AHAF incorporates dynamic α-adaptation (0.50-2.13 range) with theoretical guarantees, achieving O(1/√T) convergence (Theorem 1) and gradient saturation mitigation (Lemma 2).Evaluated on Iris and Modified National Institute of Standards and Technology (MNIST) datasets, 8-unit AHAF networks matched 128-unit Rectified Linear Unit (ReLU) models in accuracy (96.67%) while reducing parameters by 84% and improving convergence speed by 18-22%.AHAF demonstrated superior adversarial robustness under Projected Gradient Descent (PGD) attacks (ϵ=0.1),showing only 1.18% accuracy drop versus ReLU's 3.42% (p 1.0).These results challenge conventional assumptions about hybrid activation overhead, establishing AHAF as a framework for sustainable Artificial Intelligence (AI) deployment in edge computing and adversarial-prone environments.The study bridges two key paradigms -dynamic activation adaptation and minimalist architecture design -offering both theoretical advances (formalized convergence proofs) and empirical validation (energy efficiency metrics, adversarial testing).By demonstrating that small networks with meta-learned activations can outperform traditional architectures, this work provides actionable insights for developing efficient, robust AI systems without sacrificing accuracy.The replicable methodology and open benchmarks further support adoption in real-world applications where computational resources and model stability are critical constraints.enCC BYhybrid activation functionsArticle