The AWS Summit Toronto: A Day Packed with Innovation

The AWS Summit Toronto was truly one for the books! From the engaging keynote sessions to hands-on workshops, every moment was packed with insights. Spending over eight hours at the summit was absolutely worth it.

Here are some of the highlights that stood out for me:

Keynote Session with Andrew Kent

Compute

• Over 850 instance types are now available, catering to any workload.
• AWS Graviton4: a silicon processor built from scratch by AWS engineers, now powering major AWS data centers worldwide.
• AWS Trainium 2: the latest processor designed to accelerate AI workloads.

Storage

• Amazon S3 now holds over 400 trillion objects, and innovations continue with S3 Tables, introduced last year.

Databases

• With global database synchronization, a central time sync was created to reduce latency and maintain data integrity.

Inference (My Favourite)

• Amazon Bedrock empowers customers to build generative AI applications with multiple models to choose from.
• For private datasets, RAG (Retrieval-Augmented Generation) ensures AI understands your data internally.
• Bedrock Guardrails keep models operating within defined boundaries.
• Cost optimization is achieved through Model Distillation and Intelligent Prompt Routing, saving money without compromising output quality.

Customer Stories:
One story that blew me away was from Julie Levesque the VP Technology and Operations; National Bank of Canada. Her team migrated over 2,000 servers to AWS EC2 in just 9 months with zero downtime. Amazingly, she started by using her own credit card to experiment with AWS; audacious and inspiring!

EKS Framework Session

In the interactive session with AWS engineers, we explored the best practices for managing and securing EKS clusters:

Identity & Access:

• AWS Auth ConfigMap (Legacy): This method maps IAM users or roles to Kubernetes roles via a ConfigMap. It works, but it’s manual and harder to scale.
• API-Based Approach (Recommended): Using AWS IAM Authenticator or API-driven identity management allows secure, scalable, and automated access for users and service accounts.
• Cluster Access Manager: Helps restrict access to specific namespaces, ensuring least-privilege access.

Logging & Monitoring:

• Control Plane Activities: Captured via CloudTrail.
• Data Plane Activities: Captured via CloudWatch (e.g., who accessed what inside the cluster).

Security Best Practices:

• Signed Images & Immutable Tags: Ensure images deployed in production haven’t been tampered with.
• Private Endpoints: Connect pods to external systems (like S3) securely.
• SBOM (Software Bill of Materials): Documents how images are built and what packages are included, which can feed into tools like Trivy for vulnerability scanning.

Generative AI & LLMs

Data is the fuel for AI, and 80% of companies are revisiting their data strategies to improve quality and readiness for LLMs.

Amazon SageMaker AI allows building, training, fine-tuning, and deploying custom models, and you can also leverage 250+ public models.

Choosing the Right LLMs for Your Infrastructure

You don’t need to test every single model available. A practical approach is to start with 3 models and benchmark them based on your priorities. The model that performs best across your key metrics becomes the winner, guiding your infrastructure decisions.

Example:

• Suppose you have three LLMs: LLM1, LLM2, and LLM3.
• Define priorities for your benchmarks:
  • Cost: Highest priority
  • Accuracy: Medium priority
  • Speed: Lowest priority
• After testing:
  • LLM2 has the lowest cost, medium accuracy, and slowest speed. Based on your priorities, LLM2 would be the top choice.
• This approach helps align infrastructure decisions with real-world performance and business goals.

Key Requirements for Infrastructure Benchmarking:

Balance priorities: Consider performance (speed), price-performance (cost efficiency), and price (total cost to run).

Start from requirements:Define what your application needs and work backward.

Standardize testing: Use production-like input/output sizes, parallel requests, and consistent models across engines.

Measure performance metrics: Evaluate across different configurations.

Key Takeaways

Measure everything: Don’t build what you cannot measure.

Customer-driven innovation: Most AWS services exist because customer demand shaped them.

When life throws you lemons, build lemonade. Most AWS services exist because customer demand shaped them. Stones thrown at you can become stepping stones to climb higher.