AI Powered Tablet Scanner with Multilingual Voice Assistance

Medication Safety and the Need for Accurate Pill Identification

Medication safety and adherence remain major challenges in healthcare, with medication errors contributing significantly to preventable harm. A common cause of these errors is improper pill identification, particularly when medications are removed from their original packaging or when unlabeled or visually similar pills are encountered. These challenges emphasize the need for reliable pill identification tools and improved training in safe prescribing and medication management [1].

Polypharmacy and Medication Errors in Chronic Diseases

Polypharmacy is strongly associated with adverse drug events, nonadherence, drug–drug interactions, hospitalizations, and increased mortality. In nephrology, polypharmacy has been linked to medication error rates as high as 68%, highlighting the importance of medication reconciliation as a primary strategy for improving patient safety [2]. Similar challenges are observed in rheumatic diseases and the general population, where long-term combination therapy and aging-related comorbidities contribute to widespread polypharmacy [9].

Challenges Faced by Older Adults and Visually Impaired Patients

Older adults and visually impaired individuals face disproportionate risks related to medication misidentification. Studies indicate that 75–96% of elderly patients make medication-related errors, while existing solutions largely focus on healthcare providers rather than patient- centered interventions [3].

Medication management is particularly challenging for individuals with visual impairments, as traditional identification methods rely on visual cues such as pill color, shape, and imprints. These methods are often inaccessible, error-prone, and time-consuming, increasing the risk of adverse outcomes [4].

Medication Identification in Ophthalmology and Chronic Care

In chronic ophthalmic conditions such as glaucoma, medication identification remains a major challenge due to vision loss, similar packaging, and label degradation. Patients with poor visual acuity or cognitive decline often struggle to distinguish eye drop medications, leading to noncompliance and disease progression. Studies show a strong association between medication nonadherence and worsening glaucomatous visual field loss [5].

Artificial Intelligence in Modern Allopathic Medicine

AI has become integral to modern allopathic medicine, supporting diagnostics, drug discovery, patient monitoring, and clinical decision-making. From early rule-based systems such as MYCIN to contemporary deep learning models, AI has demonstrated its ability to process complex medical data and deliver personalized, evidence-based care [10].

Challenges and Advances in Pill Image Recognition

Despite progress, pill image recognition remains challenging due to variations in lighting, background, orientation, and image quality in mobile environments. Deep learning models also face constraints related to computational cost, power consumption, privacy, and reliance on cloud connectivity [12].

Lightweight, on-device solutions such as MobileDeepPill address these challenges by employing efficient multi-CNN architectures and triplet loss functions, achieving state-of-the- art performance without cloud offloading [12]. Other CNN-based approaches, including ResNet101, have demonstrated classification accuracies exceeding 98%, further supporting the role of deep learning in improving medication safety [13].

Need for Assistive and Automated Solutions

Accurate pill detection and inspection are essential for ensuring medication safety, regulatory compliance, and effective clinical care, particularly with the increasing use of automated dispensing systems [15]. Traditional manual identification methods remain time-consuming and error-prone, especially for patients with visual impairments or complex medication regimens. Consequently, AI-driven assistive technologies are increasingly recognized as critical tools for reducing medication errors and improving patient outcomes [14].

2. MATERIALS AND METHODS

System Overview

The Tablet Identification and Medicine Information System is an AI-based application developed to recognize pharmaceutical tablets using back-side image analysis and to retrieve detailed medicine information from a structured dataset. The system integrates computer vision, deep learning algorithms, and web technologies to create a complete workflow suitable for educational, research, and demonstration purposes.

The workflow consists of four main stages:

1. Image acquisition and upload through a web interface
2. Deep learning-based image classification
3. Retrieval of structured drug information from a medicine database (Excel)
4. Presentation of results through a responsive and user-friendly interface
5. Materials (Software and Tools)

2.1 Programming Language

Python 3.10: The primary programming language used for model development, backend API implementation, and data processing because of its extensive libraries for machine learning and web development.

2.2 Deep Learning Frameworks

PyTorch: Utilized for training and running the deep learning model, offering dynamic computation graphs and efficient CPU/GPU execution.

Torchvision: Used to access pretrained models such as ResNet-50 and to perform image preprocessing operations.

2.3 Model Architecture

ResNet-50 (Residual Neural Network):

• A deep convolutional neural network consisting of 50 layers.
• Employs residual connections that help prevent the vanishing gradient problem.
• Pretrained on the ImageNet dataset and later fine-tuned for tablet image classification.

2.4 Data Handling

Pandas: Used to manage and process structured medicine information stored in Excel files. OpenPyXL: Serves as the backend engine for reading .xlsx datasets.

2.5 Backend Framework

FastAPI:

• A high-performance Python framework used to build the backend API.
• Supports asynchronous request handling.
• Provides automatic JSON serialization and REST API functionality.

2.6 Frontend Technologies

HTML5: Defines the structure of the web interface.

CSS3: Responsible for layout design, styling, and responsive presentation.

JavaScript (Vanilla): Handles image uploads, API communication, and dynamic display of prediction results.

2.7 Development Environment

Ubuntu Linux: Operating system used during development.

Virtual Environment (venv): Used to isolate project dependencies. Uvicorn: ASGI server used to run the FastAPI application.

3. DATASET PREPARATION

3.1 Image Dataset

• The dataset contains back-side images of pharmaceutical tablets.
• Each class represents a specific medicine (for example Paracetamol_P650 or Flavoxate_Urivel).
• Images are arranged using a directory-based structure compatible with ImageFolder datasets.

3.2 Data Split

Training Set: Approximately 80% of the total images per class. Validation Set: Approximately 20% of the total images per class.

3.3 Image Preprocessing

• Images are resized to 224 × 224 pixels.
• Normalization is applied using ImageNet mean and standard deviation values.
• Data augmentation such as random horizontal flipping is used during training to improve model generalization.

4. METHODS

4.1 Model Training Methodology

1. A pretrained ResNet-50 model with ImageNet weights is loaded.
2. Convolutional layers are frozen to preserve previously learned visual features.
3. The final fully connected layer is modified to match the number of tablet classes.
4. Model training uses the following parameters: Loss Function: Cross-Entropy Loss

Optimizer: Adam Optimizer Learning Rate: 0.001

Training is conducted for 8 epochs while monitoring the reduction in loss values.

4.2 Prediction Method

• Uploaded images undergo the same preprocessing steps used during training.
• The trained model predicts the tablet class with the highest probability.
• This predicted class is used as a key for retrieving medicine information from the database.

4.3 Medicine Information Retrieval

• Medicine details are stored in an Excel dataset.
• Each row corresponds to a particular medicine.
• The column medicine_key functions as the primary identifier.
• Pandas is used to filter the dataset and retrieve the record corresponding to the predicted class.

4.4 Web Application Workflow

1. The user uploads a tablet image using the browser interface.
2. The image is sent to the FastAPI backend through an HTTP POST request.
3. The backend performs model inference and database retrieval.
4. A JSON response is returned to the frontend.

5. RESULTS

5.1 Model Performance

• Training loss gradually decreased from around 0.65 to 0.07, indicating successful model learning.
• The trained model accurately classified tablets used during testing.
• Predictions were most reliable when images were captured clearly and under conditions similar to the training data.

5.2 Functional Results

The system successfully:

• Identified tablets from uploaded images.
• Retrieved the appropriate medicine information from the Excel dataset.
• Displayed detailed drug information within a clean and structured user interface.

1.3 User Interface Results

• The interface developed for the application is:
• Mobile-responsive.
• Easy to read and visually organized.
• Appropriate for academic demonstrations and healthcare-related presentations.

6. DISCUSSION

6.1 Strengths of the System

• Provides a complete automated workflow from image input to structured output.
• Transfer learning reduces both training time and the need for very large datasets.
• Clear separation between prediction logic and data storage.
• Lightweight frontend implementation without heavy external frameworks.

6.2 Limitations

• System accuracy is strongly dependent on dataset quality and image diversity.
• Excel-based storage may not be suitable for large-scale deployment.
• The current system relies only on visual appearance and does not yet analyze tablet imprint text.

6.3 Practical Applications

• Educational demonstrations for pharmacy and artificial intelligence courses.
• Preliminary support for tablet identification.
• Prototype framework for hospital or pharmacy automation systems.

PREVIEW PAGE