AWS AI Practitioner — Day 6: Automation & Reproducibility
Learn: create repeatable training scripts, log hyperparameters and seeds, and store run metadata in S3.
Hands-on: parameterize a training script and rerun; confirm model artifacts change predictably.
Practice question:
Q1: Which practice improves experiment reproducibility?
A) Not recording random seeds B) Hardcoding dataset paths in notebooks C) Logging hyperparameters and random seeds and saving artifacts with run IDs D) Running training without version control
Answer: C — logging seeds, hyperparameters, and using run IDs helps reproducibility.
Daily Practice Questions (new)
Q1: What is an experiment run ID?
A) Unique identifier for a training run B) AWS account ID C) S3 bucket name D) IAM role
Answer: A — run IDs uniquely identify experiment runs.
Q2: How can you log hyperparameters programmatically?
A) Write a JSON manifest to S3 or use MLflow B) Use Route 53 C) Modify EC2 user data only D) Edit IAM policies
Answer: A — record hyperparams in manifests or tracking systems.
Q3: Why parameterize training scripts?
A) Enable reproducible reruns with different settings B) Increase costs C) Hide logs D) Disable monitoring
Answer: A — parameters allow controlled reruns and experimentation.
Q4: How should random seeds be saved?
A) Include them in run manifest and code B) Never record them C) Store in /tmp only D) Use random hardware only
Answer: A — store seeds for reproducibility.
Q5: What is a pipeline in ML workflows?
A) An automated sequence for data, training, deploy steps B) A network pipe C) A database engine D) CloudWatch log group
Answer: A — pipelines automate repeatable processes.
Q6: How to compare experiment runs?
A) Load metrics and compare key indicators (accuracy, loss) B) Compare S3 sizes only C) Rely on memory D) Delete older runs
Answer: A — compare metrics to evaluate runs.
Q7: Why avoid hardcoded paths in experiments?
A) Reduces portability and reproducibility issues B) Speeds up training C) Improves security automatically D) Is required by AWS
Answer: A — parameterized paths improve portability.
Q8: How to restore a previous model version?
A) Fetch artifact by run ID from S3 B) Rebuild training dataset only C) Use Route 53 D) Update CloudWatch
Answer: A — retrieve artifacts by run ID.
Q9: What is model drift?
A) Degradation in model performance over time B) A deployment strategy C) A storage class D) A logging level
Answer: A — drift indicates performance decline with changing data.
Q10: How to snapshot the environment for reproducibility?
A) Record package versions and runtime metadata B) Only save model weights C) Disable logging D) Use arbitrary filenames
Answer: A — snapshot dependencies and runtime info for reproducibility.
Review Questions (previous lessons)
Q1: Where should model artifacts be stored for durability?
A) S3 B) /tmp C) Browser cache D) Local logs
Answer: A — S3 is suitable for durable artifact storage.
Q2: What is a quick way to run a smoke test on an endpoint?
A) Send a sample request and verify the response B) Delete the endpoint C) Run a full training job D) Update IAM policies
Answer: A — sample requests validate basic functionality.
Q3: What differentiates batch from real-time inference?
A) Batch processes bulk offline jobs; real-time is low-latency serving B) Batch is always cheaper C) Real-time never uses containers D) Batch uses Route 53
Answer: A — batch is for bulk processing; real-time is for immediate responses.
Q4: How do you monitor endpoints on AWS?
A) CloudWatch metrics and logs B) Route 53 records C) IAM policies only D) S3 lifecycle rules
Answer: A — CloudWatch provides observability data.
Q5: Why apply least-privilege IAM policies?
A) To minimize security risks and blast radius B) To speed up networks C) To reduce storage D) To increase costs
Answer: A — minimal permissions reduce exposure.
Q6: How to estimate training/deployment costs roughly?
A) Consider instance types and uptime durations B) Check DNS settings C) Look at bucket prefixes D) Use local machine cost
Answer: A — instance pricing and run time drive cost estimates.
Q7: What practice supports reproducibility of experiments?
A) Logging seeds and run manifests B) Randomly renaming files C) Deleting logs D) Not tracking versions
Answer: A — seeds and manifests aid reproducibility.
Q8: Why parameterize training scripts?
A) To run experiments reproducibly with different hyperparams B) To avoid version control C) To hide results D) To disable logging
Answer: A — parameters make runs reproducible and configurable.
Q9: What is a canary deployment?
A) Rolling out to a small subset of traffic to validate changes B) Deleting all previous versions C) A type of S3 storage D) Changing DNS only
Answer: A — canaries reduce risk by limited rollouts.
Q10: How can you reduce logging costs?
A) Use sampling, aggregation, and retention policies B) Log everything forever C) Store logs in plaintext in repo D) Disable monitoring entirely
Answer: A — sampling and retention cut storage costs.