Google Cloud SQL for MySQL is now supported as a migration source.The new service provides cost-effective, high-performance, scalable storage for compute workloads. Besides HDFS, the service now supports Amazon FSx for Lustre.
#Pyspark to access smb share from mac windows
It's now possible to transfer data between HDFS and S3, EFS or FSx for Windows File Server. The feature is in public preview.ĭata Sync is the service for data synchronization between on-premise and cloud storage, or different cloud storage services. The datashare must be located in the data producer's Redshift, though. Subscribers can now query 3rd party data sets directly from Redshift without the need of copying the data to the clust.
#Pyspark to access smb share from mac manual
Previously, the action required manual intervention or a dedicated data pipeline. The 3rd party data subscribers can now set up an auto-export feature to bring the new revisions of the subscribed datasets to their S3 buckets. The new scheduling should work great for queues of both long-running and short-running jobs. The First-In, First-Out (FIFO) policy was completed with a fair-share policy where the service tries to allocate resources to the jobs equally, or based on the defined weights and priorities. Batch console shows Step Function workflows using Batch jobs.
Two new features in the AWS Batch service: S3 in public preview to create continuous backups or periodic snapshots of S3 buckets.Change events represent data modification, such as INSERT or CREATE TABLE whereas access events represent data reads, such as SELECT statements. Īn activity stream stores all change and access events. Athena uses Lake Formation Data Filtering to implement cell-, row- and column-level fine-grained access for ACID-compliant governed tables. The feature is currently in a public preview. In addition to the governed tables, Athena now supports Apache Iceberg as the transactional data format. Recently, this feature got extended by the possibility to query data stores of different AWS accounts. A federated query can read data stored elsewhere than in an S3 bucket. Athena connects now to Glue Data Catalog to look for the partitions that are relevant for the query execution. The plan includes fine grained details such as the CPU usage or the number of processed rows. It's available to get a more detailed view of the query execution plan. This time, I'm also trying to highlight the most important features. For the updates, I'm omitting the version upgrades which are quite frequent changes especially for the managed RDBMS services. I'm covering here the data services with some exceptions most often related to the security services.