2.4. Exceptions & Robust Error Handling
Overview
Learn to catch and handle errors cleanly so your code doesn't crash--and users see helpful messages instead of tracebacks.
Introduction: Why Robust Error Handling Matters
In the real world, things go wrong. Files get deleted, network connections fail, users enter invalid data, and APIs return unexpected responses. The difference between amateur and professional code often comes down to how gracefully it handles these inevitable failures.
Python's exception handling system gives you the tools to anticipate problems, recover from errors, and provide meaningful feedback to users. Instead of letting your program crash with a confusing traceback, you can catch errors, log useful information, and either fix the problem automatically or guide users toward a solution.
Consider these two approaches to reading a file:
❌ Fragile approach:
def read_config():
file = open('config.txt', 'r')
content = file.read()
file.close()
return content
# This crashes if config.txt doesn't exist
config = read_config()
✅ Robust approach:
def read_config():
try:
with open('config.txt', 'r') as file:
return file.read()
except FileNotFoundError:
print("Configuration file not found. Using defaults.")
return "default_setting=true"
except PermissionError:
print("Permission denied reading config file.")
return None
# This handles missing files gracefully
config = read_config()
The second version anticipates what could go wrong and handles each scenario appropriately. This is the foundation of professional Python development.
The Complete Exception Handling Structure: try, except, else, finally
Python's exception handling uses four keywords that work together to create robust error management:
Basic try/except Structure
The most common pattern uses try and except:
try:
# Code that might raise an exception
result = 10 / 0
except ZeroDivisionError:
# Code that runs if the exception occurs
print("Cannot divide by zero!")
result = None
Adding else: Code for Success
The else block runs only if no exceptions were raised in the try block:
def safe_divide(a, b):
try:
result = a / b
except ZeroDivisionError:
print("Cannot divide by zero!")
return None
else:
print(f"Division successful: {a} / {b} = {result}")
return result
# Example usage
safe_divide(10, 2) # Prints success message and returns 5.0
safe_divide(10, 0) # Prints error message and returns None
Adding finally: Cleanup Code
The finally block always runs, regardless of whether an exception occurred. This is perfect for cleanup operations:
def process_file(filename):
file = None
try:
file = open(filename, 'r')
data = file.read()
# Process the data
processed = data.upper()
return processed
except FileNotFoundError:
print(f"File {filename} not found")
return None
except PermissionError:
print(f"Permission denied for {filename}")
return None
else:
print("File processed successfully")
finally:
# This always runs, ensuring file is closed
if file and not file.closed:
file.close()
print("File closed")
Complete Structure Example
Here's a comprehensive example showing all four components:
import json
def load_user_data(user_id):
filename = f"user_{user_id}.json"
file = None
try:
file = open(filename, 'r')
data = json.load(file)
except FileNotFoundError:
print(f"User {user_id} data file not found")
return None
except json.JSONDecodeError as e:
print(f"Invalid JSON in user file: {e}")
return None
except PermissionError:
print(f"Permission denied accessing user {user_id} data")
return None
else:
# Only runs if no exceptions occurred
print(f"Successfully loaded data for user {user_id}")
return data
finally:
# Always runs - cleanup
if file and not file.closed:
file.close()
Catching Specific Exceptions
One of the biggest mistakes beginners make is using bare except: clauses that catch all exceptions. This can hide bugs and make debugging nearly impossible.
The Problem with Bare Except
# ❌ Bad: Catches everything, including KeyboardInterrupt
def risky_function():
try:
# Some operation
result = some_complex_operation()
except: # This is too broad!
print("Something went wrong")
return None
This approach masks important errors and can even prevent users from stopping your program with Ctrl+C.
Catching Specific Exception Types
Instead, catch only the exceptions you expect and know how to handle:
def convert_to_number(value):
try:
# Try integer first
return int(value)
except ValueError:
try:
# If that fails, try float
return float(value)
except ValueError:
# If both fail, it's not a valid number
print(f"'{value}' is not a valid number")
return None
# Test it
print(convert_to_number("42")) # Returns 42
print(convert_to_number("3.14")) # Returns 3.14
print(convert_to_number("hello")) # Prints error, returns None
Catching Multiple Exception Types
You can catch multiple specific exceptions at once:
def read_and_parse_file(filename):
try:
with open(filename, 'r') as file:
content = file.read()
data = json.loads(content)
return data
except (FileNotFoundError, PermissionError) as e:
print(f"File access error: {e}")
return None
except json.JSONDecodeError as e:
print(f"JSON parsing error: {e}")
return None
Using Exception Information
The as keyword captures the exception object, giving you access to error details:
def divide_numbers(numbers):
results = []
for i, num in enumerate(numbers):
try:
result = 100 / num
results.append(result)
except ZeroDivisionError as e:
print(f"Error at position {i}: {e}")
results.append(float('inf'))
except TypeError as e:
print(f"Type error at position {i}: {e} (value: {num})")
results.append(None)
return results
# Example usage
test_data = [10, 0, "hello", 5, 2.5]
results = divide_numbers(test_data)
print(results) # [10.0, inf, None, 20.0, 40.0]
Raising Exceptions
Sometimes you need to signal that an error has occurred in your own code. The raise statement lets you throw exceptions deliberately.
Basic Exception Raising
def validate_age(age):
if not isinstance(age, (int, float)):
raise TypeError("Age must be a number")
if age < 0:
raise ValueError("Age cannot be negative")
if age > 150:
raise ValueError("Age cannot exceed 150 years")
return True
# Usage
try:
validate_age("twenty") # Raises TypeError
except TypeError as e:
print(f"Type error: {e}")
try:
validate_age(-5) # Raises ValueError
except ValueError as e:
print(f"Value error: {e}")
Re-raising Exceptions
Sometimes you want to log an error but still let it propagate up:
def critical_operation():
try:
# Some operation that might fail
result = risky_database_operation()
except ConnectionError as e:
# Log the error for debugging
print(f"Database connection failed: {e}")
# But still raise it - this is critical
raise
return result
Raising with Custom Messages
You can provide more context when re-raising:
def process_user_input(data):
try:
return json.loads(data)
except json.JSONDecodeError as e:
# Add context to the original error
raise ValueError(f"Invalid user data format: {e}") from e
The from e part preserves the original exception context, which is helpful for debugging.
Custom Exception Classes
For larger applications, creating your own exception types makes error handling more precise and meaningful.
Creating Simple Custom Exceptions
class ValidationError(Exception):
"""Raised when data validation fails"""
pass
class DatabaseError(Exception):
"""Raised when database operations fail"""
pass
class AuthenticationError(Exception):
"""Raised when user authentication fails"""
pass
# Usage
def login_user(username, password):
if not username:
raise ValidationError("Username cannot be empty")
if len(password) < 8:
raise ValidationError("Password must be at least 8 characters")
# Simulate authentication check
if not authenticate(username, password):
raise AuthenticationError(f"Invalid credentials for user: {username}")
return True
Advanced Custom Exceptions with Data
Custom exceptions can carry additional information:
class ValidationError(Exception):
"""Enhanced validation error with field information"""
def __init__(self, message, field=None, value=None):
super().__init__(message)
self.field = field
self.value = value
def __str__(self):
if self.field:
return f"Validation error in '{self.field}': {self.args[0]}"
return self.args[0]
class User:
def __init__(self, username, email, age):
self.username = self._validate_username(username)
self.email = self._validate_email(email)
self.age = self._validate_age(age)
def _validate_username(self, username):
if not username or len(username) < 3:
raise ValidationError(
"Username must be at least 3 characters long",
field="username",
value=username
)
return username
def _validate_email(self, email):
if '@' not in email:
raise ValidationError(
"Email must contain @ symbol",
field="email",
value=email
)
return email
def _validate_age(self, age):
if not isinstance(age, int) or age < 0:
raise ValidationError(
"Age must be a positive integer",
field="age",
value=age
)
return age
# Usage with detailed error handling
try:
user = User("jo", "invalid-email", -5)
except ValidationError as e:
print(f"User creation failed: {e}")
print(f"Problem field: {e.field}")
print(f"Problem value: {e.value}")
Exception Hierarchies
You can create exception hierarchies for more sophisticated error handling:
class APIError(Exception):
"""Base class for API-related errors"""
pass
class APIConnectionError(APIError):
"""Connection to API failed"""
pass
class APIAuthError(APIError):
"""API authentication failed"""
pass
class APIRateLimitError(APIError):
"""API rate limit exceeded"""
def __init__(self, message, retry_after=None):
super().__init__(message)
self.retry_after = retry_after
# Now you can catch at different levels
def handle_api_request():
try:
return make_api_call()
except APIRateLimitError as e:
print(f"Rate limited. Retry after: {e.retry_after} seconds")
return None
except APIAuthError:
print("Authentication failed. Check API key.")
return None
except APIConnectionError:
print("Connection failed. Check network.")
return None
except APIError:
print("General API error occurred")
return None
Logging vs. Printing Errors
As your applications grow, printing error messages to the console becomes inadequate. Python's logging module provides a professional way to record, categorize, and manage error information.
The Problem with print()
# ❌ Not suitable for production
def process_orders(orders):
for order in orders:
try:
process_single_order(order)
except Exception as e:
print(f"Error processing order {order.id}: {e}")
Problems with this approach:
- Output gets mixed with regular program output
- No way to control verbosity in different environments
- Difficult to search or analyze errors later
- No timestamps or context information
Using Python's logging Module
import logging
# Configure logging
logging.basicConfig(
level=logging.INFO,
format='%(asctime)s - %(name)s - %(levelname)s - %(message)s',
handlers=[
logging.FileHandler('app.log'),
logging.StreamHandler() # Also log to console
]
)
logger = logging.getLogger(__name__)
def process_orders(orders):
logger.info(f"Starting to process {len(orders)} orders")
for order in orders:
try:
process_single_order(order)
logger.debug(f"Successfully processed order {order.id}")
except ValidationError as e:
logger.warning(f"Validation error for order {order.id}: {e}")
except DatabaseError as e:
logger.error(f"Database error processing order {order.id}: {e}")
except Exception as e:
logger.critical(f"Unexpected error processing order {order.id}: {e}")
# For critical errors, you might want to re-raise
raise
logger.info("Finished processing orders")
Different Logging Levels
Python's logging module provides several levels of severity:
import logging
logger = logging.getLogger(__name__)
def demonstrate_logging_levels():
logger.debug("Detailed diagnostic information")
logger.info("General information about program execution")
logger.warning("Something unexpected happened, but program continues")
logger.error("Serious problem occurred")
logger.critical("Very serious error - program may stop")
# You can set different minimum levels
logging.getLogger().setLevel(logging.WARNING) # Only show warnings and above
Structured Logging for Better Analysis
For production applications, consider structured logging:
import logging
import json
class JSONFormatter(logging.Formatter):
def format(self, record):
log_entry = {
'timestamp': self.formatTime(record),
'level': record.levelname,
'module': record.module,
'message': record.getMessage(),
'function': record.funcName,
'line': record.lineno
}
# Add exception info if present
if record.exc_info:
log_entry['exception'] = self.formatException(record.exc_info)
return json.dumps(log_entry)
# Set up JSON logging
handler = logging.StreamHandler()
handler.setFormatter(JSONFormatter())
logger = logging.getLogger(__name__)
logger.addHandler(handler)
logger.setLevel(logging.INFO)
def process_payment(amount, currency):
try:
# Payment processing logic
result = charge_credit_card(amount, currency)
logger.info("Payment processed successfully",
extra={'amount': amount, 'currency': currency})
return result
except PaymentError as e:
logger.error("Payment failed",
extra={'amount': amount, 'currency': currency, 'error': str(e)})
raise
Best Practices and Real-World Examples
1. Be Specific About What You Catch
# ✅ Good: Specific exception handling
def parse_config_file(filename):
try:
with open(filename, 'r') as f:
return yaml.safe_load(f)
except FileNotFoundError:
logger.error(f"Config file {filename} not found")
return {}
except yaml.YAMLError as e:
logger.error(f"Invalid YAML in {filename}: {e}")
return {}
except PermissionError:
logger.error(f"Permission denied reading {filename}")
return {}
# ❌ Bad: Catching everything
def parse_config_file(filename):
try:
with open(filename, 'r') as f:
return yaml.safe_load(f)
except Exception: # Too broad!
return {}
2. Fail Fast When Appropriate
Sometimes it's better to crash early rather than continue with invalid data:
def initialize_database(connection_string):
"""Initialize database connection - fail fast if it doesn't work"""
try:
conn = create_connection(connection_string)
test_connection(conn)
return conn
except DatabaseError as e:
logger.critical(f"Cannot connect to database: {e}")
# Don't try to continue - this is critical
raise SystemExit(1)
3. Provide Recovery Options
def load_user_preferences(user_id):
"""Load user preferences with fallback to defaults"""
try:
return load_from_database(user_id)
except DatabaseError:
logger.warning(f"Database unavailable for user {user_id}, using cache")
try:
return load_from_cache(user_id)
except CacheError:
logger.warning(f"Cache also unavailable for user {user_id}, using defaults")
return get_default_preferences()
4. Context Managers for Resource Management
class DatabaseTransaction:
def __init__(self, connection):
self.connection = connection
self.transaction = None
def __enter__(self):
self.transaction = self.connection.begin()
return self.transaction
def __exit__(self, exc_type, exc_val, exc_tb):
if exc_type is None:
# No exception occurred, commit
self.transaction.commit()
logger.info("Database transaction committed successfully")
else:
# Exception occurred, rollback
self.transaction.rollback()
logger.error(f"Database transaction rolled back due to: {exc_val}")
return False # Don't suppress the exception
# Usage
def transfer_money(from_account, to_account, amount):
with DatabaseTransaction(db_connection) as transaction:
# If any operation fails, transaction will be rolled back
withdraw(from_account, amount)
deposit(to_account, amount)
log_transfer(from_account, to_account, amount)
Conclusion
Robust error handling is what separates professional code from amateur scripts. By mastering Python's exception handling mechanisms, you can:
- Anticipate failures and handle them gracefully
- Provide meaningful feedback to users instead of cryptic tracebacks
- Log detailed information for debugging and monitoring
- Create hierarchies of custom exceptions for precise error management
- Build resilient applications that recover from unexpected conditions
Remember these key principles:
- Catch specific exceptions, not everything
- Use logging instead of print statements for error reporting
- Fail fast when errors are unrecoverable
- Provide fallback options when possible
- Always clean up resources in finally blocks or context managers
With these tools and techniques, your Python applications will handle the inevitable problems of the real world with grace and professionalism.
Next, let's jump into an introduction to testing!