當電子發票遇見 Google Cloud Function
•
2 likes•1,101 views
20181120-Cloud Native Forum QNAP Kai Chu Chung
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to 當電子發票遇見 Google Cloud Function
Similar to 當電子發票遇見 Google Cloud Function (20)
More from inwin stack
More from inwin stack (20)
Recently uploaded
Recently uploaded (20)
當電子發票遇見 Google Cloud Function
- 2. Hello! I am Cage Chung I am here because I like to share my experiences. You can find me at https://kaichu.io QNAP 雲端應用部資深工程師 GCPUG Taipei organizer Google Cloud Expert (GCP) 2
- 4. GCPUG Taipei & Taiwan Java User Group Meetup #42 2018/11/19 at CIT
- 5. Agenda ◇ 電子發票 ◇ Google Cloud Functions ◇ Einoive ◇ Demo ◇ Trips & Tips 5
- 6. 電子發票 Let’s start with the first set of slides 1 6
- 11. Google Cloud Functions Let’s start with the second set of slides 2 11
- 12. Cloud functions 12 beta alpha Node.js 6 Node.js 8 Node.js 10 Python 3.7 Go 1.11 alpha
- 13. How it works 13 Cloud Functions - Event-driven Serverless Computing | Cloud Functions | Google Cloud - https://cloud.google.com/functions/
- 14. Functions will be assigned as endpoint 1. POST 2. PUT 3. GET 4. DELETE 5. OPTIONS Triggers - Http 14 --trigger-http
- 15. Triggers - Bucket 15 Evey change in files in this bucket will trigger functions execution. --trigger-bucket=TRIGGER_BUCKET
- 16. Triggers - Topic 16 Name of Pub/Sub topic. Every message published in this topic will trigger function execution with message contents passed as input data. --trigger-topic=TRIGGER_TOPIC
- 17. Triggers - Event type 17 EVENT_PROVIDER EVENT_TYPE EVENT_TYPE_DEFAULT RESOURCE_TYPE RESOURCE_OPTIONAL cloud.pubsub google.pubsub.topic.publish Yes topic No cloud.pubsub providers/cloud.pubsub/eventTypes/topic.publish No topic No cloud.storage google.storage.object.archive No bucket No cloud.storage google.storage.object.delete No bucket No cloud.storage google.storage.object.finalize Yes bucket No cloud.storage google.storage.object.metadataUpdate No bucket No cloud.storage providers/cloud.storage/eventTypes/object.change No bucket No google.firebase.analytics.event providers/google.firebase.analytics/eventTypes/event.log Yes firebase analytics No google.firebase.database.ref providers/google.firebase.database/eventTypes/ref.create Yes firebase database No google.firebase.database.ref providers/google.firebase.database/eventTypes/ref.delete No firebase database No google.firebase.database.ref providers/google.firebase.database/eventTypes/ref.update No firebase database No google.firebase.database.ref providers/google.firebase.database/eventTypes/ref.write No firebase database No google.firestore.document providers/cloud.firestore/eventTypes/document.create Yes firestore document No google.firestore.document providers/cloud.firestore/eventTypes/document.delete No firestore document No google.firestore.document providers/cloud.firestore/eventTypes/document.update No firestore document No google.firestore.document providers/cloud.firestore/eventTypes/document.write No firestore document No
- 18. gcloud functions 18 gcloud functions deploy (NAME : --region=REGION) [--entry-point=ENTRY_POINT] [--memory=MEMORY] [--retry] [--runtime=RUNTIME] [--source=SOURCE] [--stage-bucket=STAGE_BUCKET] [--timeout=TIMEOUT] [--update-labels=[KEY=VALUE,...]] [--clear-labels | --remove-labels=[KEY,...]] [--trigger-bucket=TRIGGER_BUCKET | --trigger-http | --trigger-topic=TRIGGER_TOPIC | --trigger-event=EVENT_TYPE --trigger-resource=RESOURCE] [GCLOUD_WIDE_FLAG ...]
- 19. Einvoice Let’s start with the third set of slides 3 19
- 20. 消費發票 彙整通知 20
- 25. Three steps Authorization Gmail / Drive / Spreadsheet Initialize watch to get notification on new emails Processing and acting on drive spreadsheet 25
- 26. 26 Authorize access to G Suite data Cloud Functions Cloud Storage Save token Get gmail permission token
- 27. 27 exports.oauth2init = (req, res) => { // Define OAuth2 scopes const scopes = [ 'https://www.googleapis.com/auth/gmail.readonly, 'https://www.googleapis.com/auth/spreadsheets', 'https://www.googleapis.com/auth/drive.file', ]; // Generate + redirect to OAuth2 consent form URL const authUrl = oauth.client.generateAuthUrl({ access_type: 'offline', scope: scopes, prompt: 'consent' // Required in order to receive a refresh token every time }); return res.redirect(authUrl); };
- 28. 28 exports.oauth2callback = (req, res) => { // Get authorization code from request const code = req.query.code; // OAuth2: Exchange authorization code for access token return new Promise((resolve, reject) => { oauth.client.getToken(code, (err, token) => (err ? reject(err) : resolve(token)) ); }).then((token) => { // Get user email (to use as a Datastore key) oauth.client.credentials = token; return Promise.all([token, oauth.getEmailAddress()]); }) .then(([token, emailAddress]) => { // Store token in Datastore return Promise.all([ emailAddress, oauth.saveToken(emailAddress) ]); }) .then(([emailAddress]) => { // Respond to request res.redirect(`/initWatch?emailAddress=${querystring.escape(emailAddress)}`); }) .catch((err) => { // Handle error console.error(err); res.status(500).send('Something went wrong; check the logs.'); }); };
- 29. 29 Initialize a ‘watch’ for Gmail changes Cloud Functions Initialize watch Cloud Pub/Sub Watch notification On new message
- 30. 30 exports.initWatch = (req, res) => { // Require a valid email address if (!req.query.emailAddress) { return res.status(400).send('No emailAddress specified.'); } const email = querystring.unescape(req.query.emailAddress); if (!email.includes('@')) { return res.status(400).send('Invalid emailAddress.'); } // Retrieve the stored OAuth 2.0 access token return oauth.fetchToken(email) .then(() => { // Initialize a watch return pify(gmail.users.watch)({ auth: oauth.client, userId: 'me', resource: { labelIds: ['INBOX'], topicName: config.TOPIC_NAME } }); })
- 31. 31 .then(helpers.getOrCreateEinvoiceFolder) .then(data => { console.log(`${email} einvoice folder ${data.folderName}(${data.id})`); oauth.saveEinvoiceFolder(email, data) }) .then(() => { // Respond with status res.write(`Watch & Create Drive initialized!`); res.status(200).end(); }) .catch((err) => { // Handle errors if (err.message === config.UNKNOWN_USER_MESSAGE) { res.redirect('/oauth2init'); } else { console.error(err); res.status(500).send('Something went wrong; check the logs.'); } }); };
- 32. 32 Process Cloud Functions Cloud Pub/Sub Watch notification On new message Get new message Check drive folder Insert or create spreadsheet
- 33. 33 { "id": "166e11f5ace429e2", "threadId": "166e11f5ace429e2", "labelIds": [ "IMPORTANT", "CATEGORY_UPDATES", "INBOX" ], "payload": { "headers": [ { "name": "From", "value": "einvoice@fia.gov.tw" }, { "name": "Subject", "value": "財政部電子發票整合服u200b務平台-消費發票彙整通知，手機條碼:/HO**BPQ(每週)" }, ],
- 34. 34 "parts": [ { "partId": "1", "mimeType": "application/octet-stream", "filename": "093769936901.csv", "headers": [ { "name": "Content-Type", "value": "application/octet-stream; name=093769936901.csv" }, { "name": "Content-Transfer-Encoding", "value": "base64" }, { "name": "Content-Disposition", "value": "attachment; filename=093769936901.csv" } ], "body": { "attachmentId": "ANGjdJ-fUlJcaxIJwF8oxUwagS5M…7MTGfdDvu", "size": 13737 } } ] }, }
- 35. 35 exports.onNewMessage = (event) => { // Parse the Pub/Sub message const dataStr = Buffer.from(event.data, 'base64').toString('ascii'); const dataObj = JSON.parse(dataStr); return oauth.fetchToken(dataObj.emailAddress) .then(helpers.listMessageIds) .then(res => helpers.getMessageById(res.messages[0].id)) // Most recent message .then(msg => helpers.isValidEinvoiceFormat(msg)) .then(msg => Promise.all([msg, oauth.fetchEinvoiceFolder(dataObj.emailAddress), helpers.getAllCSV(msg)])) .then(([msg, einvoice, csv]) => Promise.all([msg, einvoice, helpers.getCSVRows(csv)])) .then(([msg, einvoice, [filename, rows]]) => Promise.all([msg, helpers.getOrCreateSpreadsheet(einvoice.id, filename), rows])) .then(([msg, spreadsheetId, rows]) => Promise.all([msg, helpers.saveToSpreadsheet(spreadsheetId, rows)])) .then(([msg]) => { console.log(`onNewMessage(${msg.id}) done.`) }) .catch((err) => { // Handle unexpected errors if (!err.message || err.message !== config.NO_LABEL_MATCH) { console.error(err); } }); };
- 36. Demo4 36
- 37. Trips & Tips Let’s start with the fifth set of slides 5 37
- 38. “ Cloud Pub/Sub requires that you grant Gmail privileges to publish notifications to your topic. 38
- 40. “ 消費發票彙整通知附件為 ANSI(ISO-8859-1、BIG5、MS950) 編碼，需 要額外進行 decode 40 exports.getCSVRows = ([csv]) => { // decode csv to big5 encoding const text = iconv.decode(csv, 'Big5'); const rows = text.split('n').map(row => row.split('|')); const filename = rows[0][3].substr(0, 6); return new Promise(resolve => resolve([filename, rows])); };
- 41. “exports.isValidEinvoiceFormat = (msg) => { return new Promise((resolve, reject) => { const subject = msg.payload.headers.filter(h => h.name === 'Subject'); if (subject.length && subject[0].value.indexOf('財政部電子發票整合服 務平台-消費發票彙整通知，手機條碼 ') > -1) { resolve(msg) } }) }; Custom message filter 41
- 42. “ Custom message filter 42 exports.isValidEinvoiceFormat = (msg) => { return new Promise((resolve, reject) => { const from = msg.payload.headers.find(h => h.name === 'From'); if (["einvoice@fia.gov.tw"].indexOf(from.value) > -1) { resolve(msg) } }) };
- 43. “ Cloud Functions Node.js Emulator 43 npm install -g @google-cloud/functions-emulator alpha
- 44. “ 44 pify(sheets.spreadsheets.batchUpdate)({ auth: oauth.client, spreadsheetId, resource: { "requests": [ { "autoResizeDimensions": { "dimensions": { "sheetId": 0, "dimension": "COLUMNS", "startIndex": 0, "endIndex": 10 } } } ] }, }).then((res) => { console.log('saveToSpreadsheet done.') }) Auto format
- 45. References 1. Using serverless on GCP to add custom intelligence to Gmail | Google Cloud Blog 2. Push Notifications | Gmail API | Google Developers 3. API Reference | Gmail API | Google Developers 4. API Reference | Drive REST API | Google Developers 5. Google Sheets API | Sheets API | Google Developers 6. Cloud Functions Node.js Emulator | Cloud Functions Documentation | Google Cloud 45
- 46. Thanks! Any questions? You can find me at: ◇ https://kaichu.io ◇ cage.chung@gmail.com 46