Cost-Optimized Analytics Challenge
Splunk Enterprise is powerful for data analysis and proof-of-concept work, but traditional deployments incur continuous costs even when not in use. Development and testing environments need the full Splunk capability without the operational overhead of persistent infrastructure.
This project provides automated infrastructure for deploying ephemeral Splunk Enterprise instances with true fresh install approaches and complete cost optimization.
Ephemeral Infrastructure Architecture
True Fresh Install Approach
- EC2 t3.large with Amazon Linux and automated Splunk installation
- Complete deploy/destroy cycles with zero idle costs when not in use
- 100GB gp3 EBS with delete-on-termination for cost optimization
- SSM Session Manager access with port forwarding for secure connectivity
Cost Management Strategy
- Zero idle costs - no infrastructure exists when destroyed
- Active costs approximately $0.08/hour for t3.large + EBS prorated
- Typical 3-hour session costs approximately $0.25
- Annual usage (weekly 3-hour sessions) approximately $13/year
Operational Automation
- Automated installation through comprehensive user data scripts
- CloudWatch Logs integration for installation monitoring and debugging
- Cost alarms at $5/$10/$20 thresholds for budget protection
- Start/stop scripts for session management during active use
Strategic Value for Development
This isn’t just Splunk hosting - it’s cost-conscious infrastructure for analytics experimentation. The ephemeral approach enables teams to use enterprise-grade analytics tools without ongoing operational costs.
Development Benefits
- Full Splunk Enterprise capability for proof-of-concept work
- No ongoing costs when not actively developing or analyzing
- Rapid deployment from zero to fully functional Splunk instance
- Secure access through AWS SSM without inbound network exposure
Security & Compliance
- No inbound access - all connectivity through SSM Session Manager
- Outbound HTTPS only for SSM communication and software downloads
- CloudWatch Logs capture all installation and operational activities
- Instance isolation with dedicated VPC and security group configuration
Integration Patterns
- Backend configuration integration with foundation-terraform-bootstrap
- Deployment roles integration with terraform-aws-deployment-roles
- SSM Parameter Store outputs for consuming project discovery
- Consistent project patterns following established infrastructure conventions
This infrastructure demonstrates innovative cost optimization while maintaining enterprise-grade functionality for analytics and development workflows.
Repository: service-ephemeral-splunk