Continue Integration and Continue Delivery

CICD – Blog by John Chang

Continue Integration and Continue Delivery

這個月專注於幫客戶的專案建構自動化的部屬流程。

目前部署的情況是,由開發商將source code打包好後上傳至S3 bucket,再由我們的IT人員把檔案抓下來,接著就是用command line搭配shell script進行某台機器的更新。繼續閱讀

Data Processing II

(只提供英文版本)

Big Data – Blog by YY

Data Processing II

In the previous part of the series, we looked at various data processing techniques, from traditional (Batching processing) to the one (Hybrid-processing) commonly being used nowadays. Besides, in the previous series, we only focused on “INPUT.” However, “OUTPUT” part, is what upper level managers care about, because it involved how the INPUT being calculated, what fields in INPUTs being joined, and, etc regarding the numbers/analytics charts in reports, as well as the index that indicates the status in organizations.

繼續閱讀

Data Processing

(只提供英文版本)

Big Data – Blog by YY

Data Processing

Architecture Design, as the heart in every aspect, is a complex task.

For example, the architecture design in automobile, the one in skyscraper, and etc. The reason is the architect needs to take everything, if possible, into consideration.

繼續閱讀

Continue Integration and Continue Delivery

CICD – Blog by John Chang

Continue Integration and Continue Delivery

CI/CD跟DevOps其實是密不可分的,CI想解決的問題是,傳統的開發模式,每個開發者都是在自己的電腦上架設開發環境,等完成一個段落後才commit到verison control server上,例如Git。

繼續閱讀

Data Profiling

(只提供英文版本)

Big Data – Blog by YY

Data Profiling

In the previous weekly posts, Data Governance and Data Quality, as criterion not just for earlier data processing flows, but also for afterwards analysis and consistency on data source in data warehouse, have been introduced. However, there still have minor terms in existence of Data Governance and Data Quality. and that is what will be mentioned in this post, Data Profiling, statistical criterion on raw data or processed one.

繼續閱讀