Big Data – Blog by YY
Limitations on QuickSight
In the previous blog, I mentioned how to do data analysis in between QuickSight, data visualization service, and, S3, where the sources coming from. Like other services, this kind of integration has pitfall as well. I am gonna list pitfall regarding integrating QuickSight and S3 for off the wrong route in development.
Files, the source being digested from S3 into QuickSight should be in the compliance.- Every single file MUST less than 1GB
– Total files being specified in manifest file MUST less than 10 GB
– Total number of files being specified in manifest file MUST less than 1000
– For files being stored as csv, up to 200 columns.
Data Type, the following primitive data types are in support.
– Date, JODA date formats
– Decimal, up to 4 decimal places to the right of the decimal point.
example, say, 123.45678, the decimal.
ONLY 123.4567 being popped up in QuickSight.
With the quick grasp of these limitations on QuickSight and S3 in between, it will be beneficial for developer to
anticipate all the consequences before ETL.