Learn Azure Stream Analytics for real-time data

1. Your SQL Man (I) Pvt Ltd. 1
Azure Stream Analytics

2. Your SQL Man (I) Pvt Ltd. 2
Overview
•Real-time event processing engine
•Easy to Set up
•Stream Analytics Job
•Scale/Speed

3. Your SQL Man (I) Pvt Ltd. 3
Scenarios
•Personalized & Real-time stock-trading analysis
•Alerts offered by financial services companies
•Real time fraud detection
•Data & identify protection services
•Data generated by sensors & actuators
•Web clickstream
•CRM applications

4. Your SQL Man (I) Pvt Ltd. 4
Key Capabilities
•Ease of use
•Scalability
•Reliability, Repeatability and Quick recovery
•Low cost
•Reference data
•Connectivity

5. Your SQL Man (I) Pvt Ltd. 5
Pricing

6. Your SQL Man (I) Pvt Ltd. 6
Steps to follow
•Create a Job
•Add Inputs
•Add Outputs
•Write Queries
•Run a Job

7. Your SQL Man (I) Pvt Ltd. 7
Scenario – Twitter Sentiment
•Input – Event Hub
•Event Generator – Twitter Client Application
•Query – SQL Model
•Output - Table

8. Your SQL Man (I) Pvt Ltd. 8
Stream Analytics
•Features
• Monitor
• Inputs
• Outputs
• Query
• Scale
• Configure
• Functions

9. Your SQL Man (I) Pvt Ltd. 9
Query - Windowing
•Tumbling Windows

10. Your SQL Man (I) Pvt Ltd. 10
Query - Windowing
•Hopping Windows

11. Your SQL Man (I) Pvt Ltd. 11
Query - Windowing
•Sliding Windows

12. Your SQL Man (I) Pvt Ltd. 12
Scenario – Real-time fraud detection
•Input – Event Hub
•Event Generator – Telco Generator
•Query – SQL Model
•Output - Table

13. Your SQL Man (I) Pvt Ltd. 13
Demo
Stream Analytics

14. Your SQL Man (I) Pvt Ltd. 14
Scenario – ML
•Input – Blob
•Function – ML Model
•Query – SQL Model
•Output - Blob

15. Your SQL Man (I) Pvt Ltd. 15
ML

16. Your SQL Man (I) Pvt Ltd. 16
Query Language
•Built-in Functions
•Data Types
•Query Language Elements
•Time Management
•Windowing

17. Your SQL Man (I) Pvt Ltd. 17
Built-in Functions
•Aggregate Functions
•Analytic Functions
•Record Functions
•Scalar Functions

18. Your SQL Man (I) Pvt Ltd. 18
Built-in Functions
•Aggregate Functions
• AVG
• COUNT
• MAX
• MIN
• STDEVP
• STDEV
• VAR
• VARP
• TopOne
• CollectTOP

19. Your SQL Man (I) Pvt Ltd. 19
Analytic Functions
•ISFIRST - Returns 1 if the event is the first event within a given duration, or
0 otherwise.
•LAG
•LAST
SELECT deviceid, reading, ISFIRST(mi, 10) OVER (PARTITION BY deviceid) as first FROM Input
SELECT sensorId, LAG(reading) OVER (PARTITION BY sensorId LIMIT DURATION(hour, 1) WHEN reading IS NOT NULL) FROM input
SELECT sensorId, LAST(reading) OVER (PARTITION BY sensorId LIMIT DURATION(hour, 1) WHEN reading IS NOT NULL) FROM input

20. Your SQL Man (I) Pvt Ltd. 20
Record Functions
•GetRecordProperties
• Returns a dataset with record property names and values. The result of the
GetRecordProperties function must be used with the CROSS APPLY operator
•GetRecordPropertyValue
• Returns the record value associated with the specified property.
• s
SELECT recordProperty.PropertyName, recordProperty.PropertyValue FROM input as event CROSS APPLY GetRecordProperties(event.recordField) AS recordProperty
SELECT input.DeviceID, thresholds.SensorName FROM input JOIN thresholds ON input.DeviceId = thresholds.DeviceId WHERE
GetRecordPropertyValue(input.SensorReading, thresholds.SensorName) > thresholds.Value

21. Your SQL Man (I) Pvt Ltd. 21
Scalar Functions
•CAST – CAST (fieldname as XXXX)
•GetType – Data type name of the value
•TRY_CAST
•DATENAME, DATEPART, DAY, MONTH, YEAR, DATEDIFF, DATEADD
•DATETIMEFROMPARTS

22. Your SQL Man (I) Pvt Ltd. 22
Scalar Functions
•Mathematical Function
• ABS, CEILING, EXP, FLOOR, POWER, SQUARE, SIGN, SQRT
•String Function
• LEN, CONCAT, CHARINDEX, LOWER, SUBSTRING, PATINDEX, UPPER

23. Your SQL Man (I) Pvt Ltd. 23
Data Types

24. Your SQL Man (I) Pvt Ltd. 24
Query Language Elements
• APPLY
• CASE
• CREATE TABLE
• FROM
• GROUP BY
• HAVING
• INTO
• JOIN
• SELECT
• UNION
• WHERE
• WITH

25. Your SQL Man (I) Pvt Ltd. 25
Time Management
•System.Timestamp
•TIMESTAMP BY

26. Your SQL Man (I) Pvt Ltd. 26
Demo
Stream Analytics