Continue Integration and Continue Delivery

(Provide Chinese Version only)

CICD – Blog by John Chang

Continue Integration and Continue Delivery

這個月專注於幫客戶的專案建構自動化的部屬流程。

目前部署的情況是,由開發商將source code打包好後上傳至S3 bucket,再由我們的IT人員把檔案抓下來,接著就是用command line搭配shell script進行某台機器的更新。Continue reading

Data Processing II

Big Data – Blog by YY

Data Processing II

In the previous part of the series, we looked at various data processing techniques, from traditional (Batching processing) to the one (Hybrid-processing) commonly being used nowadays. Besides, in the previous series, we only focused on “INPUT.” However, “OUTPUT” part, is what upper level managers care about, because it involved how the INPUT being calculated, what fields in INPUTs being joined, and, etc regarding the numbers/analytics charts in reports, as well as the index that indicates the status in organizations.

Continue reading

Data Processing

Big Data – Blog by YY

Data Processing

Architecture Design, as the heart in every aspect, is a complex task.

For example, the architecture design in automobile, the one in skyscraper, and etc. The reason is the architect needs to take everything, if possible, into consideration.

Continue reading

Continue Integration and Continue Delivery

(Provide Chinese Version only)

CICD – Blog by John Chang

Continue Integration and Continue Delivery

CI/CD跟DevOps其實是密不可分的,CI想解決的問題是,傳統的開發模式,每個開發者都是在自己的電腦上架設開發環境,等完成一個段落後才commit到verison control server上,例如Git。

Continue reading

Data Profiling

Big Data – Blog by YY

Data Profiling

In the previous weekly posts, Data Governance and Data Quality, as criterion not just for earlier data processing flows, but also for afterwards analysis and consistency on data source in data warehouse, have been introduced. However, there still have minor terms in existence of Data Governance and Data Quality. and that is what will be mentioned in this post, Data Profiling, statistical criterion on raw data or processed one.

Continue reading