SlideShare una empresa de Scribd logo
1 de 135
Descargar para leer sin conexión
關於測試
我說的其實是......
Hugo
@AgileCommunity.tw
先調查⼀一下
• 誰會寫 Python 程式
• 誰玩過 BDD (Behavior-driven Development)
• 誰玩過 TDD (Test-driven Development)
• 誰玩過 CI (Continuous Integration)
故事是這樣的...
有⼀一天,PM有個想法
I have a DREAM!
網站上提供
計算機的服務
功能 (Feature)
網路計算機
(Web Calculator)
1+1=2
雲端技術
⼯工程計算機要跟⼿手機
App結合
可以產⽣生
⼤大數據匯率轉換
⼀一個功能,
各⾃自表述。
使⽤用者故事 (User Story)
As a student of primary school
In order to finish my homework
I want to do arithmetic operations
使⽤用者故事
提供問題的脈絡1+1=2
規格書
• 滿⾜足四則運算
規格書 part2
• 滿⾜足四則運算
• 運算⼦子優先順序
• 交換律
• 結合律
• 分配律
規格書 part 3
• 滿⾜足四則運算
• 運算⼦子優先順序: 先括號,再× ÷,後 + −
• 交換律: x∗y = y∗x ∀ x,y ∈ S
• 結合律:(x∗y)∗z = (x∗y)∗z ∀ x,y,z ∈ S
• 分配律: x∗(y+z) = (x∗y)+(x∗z) ∀ x,y,z ∈ S
圖⽚片來源 http://goo.gl/sKZQGX
能不能舉例說明?
關於⾏行為驅動開發
(Behave Driven Development, BDD)
場景 (Scenario)
Scenario Outline: do simple operations
Given I enter <expression>
When I press "=" button
Then I get the answer <answer>
Examples:
| expression | answer |
| 3 + 2 | 5 |
| 3 - 2 | 1 |
| 3 * 2 | 6 |
| 3 / 2 | 1.5 |
| 3 +-*/ 2 | Invalid Input |
| hello world | Invalid Input |
加法/乘法交換律
Scenario Outline: satisfy commutative property
When I enter <expression1> first
And I enter <expression2> again
Then I get the same answer
Examples:
| expression1 | expression2 |
| 3 + 4 | 4 + 3 |
| 2 * 5 | 5 * 2 |
加法/乘法結合律
Scenario Outline: satisfy associative property
When I enter <expression1> first
And I enter <expression2> again
Then I get the same answer
Examples:
| expression1 | expression2 |
| (2 + 3) + 4 | 2 + (3 + 4) |
| 2 * (3 * 4) | (2 * 3) * 4 |
乘法左/右分配律
Scenario Outline: satisfy distributive property
When I enter <expression1> first
And I enter <expression2> again
Then I get the same answer
Examples:
| expression1 | expression2 |
| 2 * (1 + 3) | (2*1) + (2*3) |
| (1 + 3) * 2 | (1*2) + (3*2) |
RD: 為什麼不測試這個?
Scenario Outline: parse an expression
Given I enter <expression>
When I press "=" button
Then I get an <array>
Examples:
| expression | array |
| 1+2 | ['1','+','2'] |
| 1*2 | ['1','*','2'] |
關於測試
我說的其實是......
“Because designing the technical
solution is not the purpose of the
specification, you should focus
only on writing scenarios that
relate to the business rules.”
- Executable Specification with Scrum
QA: 

驗收測試,讓專業的來
執⾏行測試
$ python manage.py behave --dry-run
…(略)
You can implement step definitions for undefined steps with these
snippets:
@given(u'I enter "3+2"')
def step_impl(context):
raise NotImplementedError(u'STEP: Given I enter "3+2"')
@when(u'I press "=" button')
def step_impl(context):
raise NotImplementedError(u'STEP: When I press "=" button')
@then(u'I get the answer "5"')
def step_impl(context):
raise NotImplementedError(u'STEP: Then I get the answer "5"')
重構步驟
@given(u'I enter {expr}')
def step_impl(context, expr):
raise NotImplementedError(u'STEP: Given I enter {expr}')
@when(u'I press "=" button')
def step_impl(context):
raise NotImplementedError(u'STEP: When I press "="
button')
@then(u'I get the answer {answer}')
def step_impl(context, answer):
raise NotImplementedError(u'STEP: Then I get the answer
{answer}')
…(略)
複製貼上,修修改改
執⾏行測試
$ python manage.py behave
Creating test database for alias 'default'...
Feature: Web calculator # features/calc.feature:3
As a student
In order to finish my homework
I want to do arithmatical operations
Scenario Outline: do simple operations -- @1.1
Given I enter 3 + 2
Traceback (most recent call last):
...(略)
NotImplementedError: STEP: Given I enter {expr}
When I press "=" button
Then I get the answer 5
溫馨提⽰示:還沒拿掉 NotImplementError
重構步驟
from calc.calculator import Calculator
@given(u'I enter {expr}')
def step_impl(context, expr):
context.expr = expr
@when(u'I press "=" button')
def step_impl(context):
calc = Calculator()
context.answer = calc.evalString(context.expr)
@then(u'I get the answer {answer}')
def step_impl(context, answer):
assert context.answer == answer
假裝類別存在
假裝有個⽅方法
執⾏行測試
$ python manage.py behave
Creating test database for alias 'default'...
Exception ImportError: No module named 'calc.calculator';
'calc' is not a package
Traceback (most recent call last):
...(略)
File "/home/vagrant/myWorkspace/demo/features/steps/
calc.py", line 1, in <module>
from calc.calculator import Calculator
ImportError: No module named 'calc.calculator'; 'calc' is not
a package
溫馨提⽰示:還沒實作 calc.calculator
拉出介⾯面
#file: calc/calculator.py
class Calculator:
def evalString(self, string):
return 0
回傳的預設值
執⾏行測試
$ python manage.py behave
Creating test database for alias 'default'...
Feature: Web calculator # features/calc.feature:3
As a student
In order to finish my homework
I want to do arithmatical operations
Scenario Outline: do simple operations -- @1.1
Given I enter 3 + 2
When I press "=" button
Then I get the answer 5
Traceback (most recent call last):
...(略)
File "features/steps/calc.py", line 19, in step_impl
assert context.answer == answer
AssertionError
QA: 接下來就是 RD 的事了
關於測試
我說的其實是......
通過場景轉換成驗收測試,
規格變成⼀一份可執⾏行的活⽂文件。
Acceptance TestScenario
User Story
Requirement
Specification
confirmsillustrates
executes
有些⼈人熱愛中華⽂文化
圖⽚片來源 http://goo.gl/BQkP82
說中⽂文也⾏行
場景⼤大綱: 做簡單的運算
假設< 我輸⼊入<expression>
當< 我按下等號按鈕
那麼< 我得到的答案是<answer>
例⼦子:
| expression | answer |
| 3 + 2 | 5 |
| 3 - 2 | 1 |
| 3 * 2 | 6 |
| 3 / 2 | 1.5 |
| 3 +-*/ 2 | Invalid Input |
| hello world | Invalid Input |
執⾏行測試
$ python manage.py behave --dry-run --include
zh_calc
功能: 網⾴頁計算機 # features/zh_calc.feature:2
⾝身為⼀一個學⽣生
為了完成家庭作業
我想要做算術運算
場景⼤大綱: 做簡單的運算 -- @1.1
假設 < 我輸⼊入3 + 2
當 < 我按下等號按鈕
那麼 < 我得到的答案是5
…(略)
關於測試
我說的其實是......
BDD 不只是測試框架,
更是溝通需求的哲學。
關於測試驅動開發
(Test Driven Development, TDD)
單元測試 (Unit Test)
from django.test import TestCase
from calc.calculator import Calculator
# Create your tests here.
class TestCalculator(TestCase):
def setUp(self):
self.calc = Calculator()
def test_evalString(self):
evalString = self.calc.evalString
self.assertEqual(evalString('0'), 0)
執⾏行測試
$ python manage.py test -v2
test_evalString (calc.tests.TestCalculator) ... ok
--------------------------------------------------------------
Ran 1 test in 0.002s
OK
該提交程式碼與測試了
記得版本控制
$ git init
$ git add .
$ git commit -m "init project"
再多⼀一點點測試
from django.test import TestCase
from calc.calculator import Calculator
# Create your tests here.
class TestCalculator(TestCase):
def setUp(self):
self.calc = Calculator()
def test_evalString(self):
evalString = self.calc.evalString
self.assertEqual(evalString('0'), 0)
self.assertEqual(evalString('1'), 1)
Baby step,每次前進⼀一⼩小步
執⾏行測試
$ python manage.py test -v2
...(略)
test_evalString (calc.tests.TestCalculator) ... FAIL
==============================================================
FAIL: test_evalString (calc.tests.TestCalculator)
--------------------------------------------------------------
Traceback (most recent call last):
File "/home/vagrant/myWorkspace/demo/calc/tests.py", line
13, in test_evalString
self.assertEqual(evalString('1'), 1)
AssertionError: 0 != 1
溫馨提⽰示:接著完善 evalString()
關於測試
我說的其實是......
測試先⾏行,
它會告訴你下⼀一步該怎麼⾛走。
RD:
準備開⼯工啦~
待辦清單
• 數字求值
• 錯誤處理
• 處理簡單數學運算
• 處理先乘除後加減
• 處理括號運算
• 交換律、結合律、分配律
• 確認驗收測試通過
3 + 2 − 1 = (3 + 2) − 1
[[‘3’, ‘2’, ‘+’], ‘1’, ‘−’]
[‘3’, ‘2’, ‘+’, ‘1’, ‘−’]
3 + 2 × 1 = 3 + (2 × 1)
[‘3’, [‘2’, ‘1’, ‘×’], ‘+’]
[‘3’, ‘2’, ‘1’, ‘×’, ‘+’]
解析⼆二元運算
3 2 1
+
×
3 2 1
+
-
待辦清單
• 數字求值:解析字串、解析 exprStack
• 錯誤處理
• 處理簡單數學運算
• 處理先乘除後加減
• 處理括號運算
• 交換律、結合律、分配律
• 確認驗收測試通過
增加測試
from django.test import TestCase
from calc.calculator import Calculator
# Create your tests here.
class TestCalculator(TestCase):
def setUp(self):
self.calc = Calculator()
…(略)
def test_parseString(self):
parseString = self.calc.parseString
self.assertEqual(parseString('0'), ['0'])
self.assertEqual(parseString('1'), ['1'])
新增解析字串測試
執⾏行測試
$ python manage.py test -v2
...(略)
==============================================================
ERROR: test_parseString (calc.tests.TestCalculator)
--------------------------------------------------------------
Traceback (most recent call last):
File "/home/vagrant/myWorkspace/demo/calc/tests.py", line
11, in test_parseString
parseString = self.calc.parseString
AttributeError: 'Calculator' object has no attribute
'parseString'
溫馨提⽰示:接著實作解析字串的⽅方法
進⾏行實作
class Calculator:
def __init__(self):
self.exprStack = []
integer = Word(nums)
self.expr = integer + StringEnd()
def parseString(self, string):
self.exprStack = []
return self.expr.parseString(string).asList()
…(略)
integer :: '0'...'9'*
expr :: integer
執⾏行測試
$ python manage.py test -v2
...(略)
test_evalString (calc.tests.TestCalculator) ... FAIL
test_parseString (calc.tests.TestCalculator) ... ok
待辦清單
• 數字求值:解析字串、解析 exprStack
• 錯誤處理
• 處理簡單數學運算
• 處理先乘除後加減
• 處理括號運算
• 交換律、結合律、分配律
• 確認驗收測試通過
增加測試
from django.test import TestCase
from calc.calculator import Calculator
# Create your tests here.
class TestCalculator(TestCase):
def setUp(self):
self.calc = Calculator()
…(略)
def test_evalStack(self):
evalStack = self.calc.evalStack
self.assertEqual(evalStack(['0']), 0)
self.assertEqual(evalStack(['1']), 1)
新增從 stack 求值的測試
執⾏行測試
$ python manage.py test
...(略)
==============================================================
ERROR: test_evalStack (calc.tests.TestCalculator)
--------------------------------------------------------------
Traceback (most recent call last):
File "/home/vagrant/myWorkspace/demo/calc/tests.py", line
16, in test_evalStack
evalStack = self.calc.evalStack
AttributeError: 'Calculator' object has no attribute
'evalStack'
溫馨提⽰示:接著實作從 stack 求值的⽅方法
進⾏行實作
class Calculator:
def __init__(self):
self.exprStack = []
def pushStack(s, l, t):
self.exprStack.append(t[0])
integer = Word(nums).addParseAction(pushStack)
self.expr = integer + StringEnd()
def evalStack(self, stack):
op = stack.pop()
return float(op)
…(略)
從 stack 取值後回傳
執⾏行測試
$ python manage.py test -v2
...(略)
test_evalStack (calc.tests.TestCalculator) ... ok
test_evalString (calc.tests.TestCalculator) ... FAIL
test_parseString (calc.tests.TestCalculator) ... ok
待辦清單
• 數字求值:解析字串、解析 exprStack
• 錯誤處理
• 處理簡單數學運算
• 處理先乘除後加減
• 處理括號運算
• 交換律、結合律、分配律
• 確認驗收測試通過
沿⽤用測試
from django.test import TestCase
from calc.calculator import Calculator
# Create your tests here.
class TestCalculator(TestCase):
def setUp(self):
self.calc = Calculator()
…(略)
def test_evalString(self):
evalString = self.calc.evalString
self.assertEqual(evalString('0'), 0)
self.assertEqual(evalString('1'), 1)
沿⽤用先前字串求值的測試
執⾏行測試
$ python manage.py test -v2
...(略)
test_evalString (calc.tests.TestCalculator) ... FAIL
==============================================================
FAIL: test_evalString (calc.tests.TestCalculator)
--------------------------------------------------------------
Traceback (most recent call last):
File "/home/vagrant/myWorkspace/demo/calc/tests.py", line
13, in test_evalString
self.assertEqual(evalString('1'), 1)
AssertionError: 0 != 1
溫馨提⽰示:接著完善 evalString⽅方法
進⾏行實作
class Calculator:
def __init__(self):
...(略)
def parseString(self, string):
...(略)
def evalStack(self, stack):
...(略)
def evalString(self, string):
self.parseString(string)
return self.evalStack(self.exprStack)
介⾯面不做複雜的⼯工作
執⾏行測試
$ python manage.py test -v2
...(略)
test_evalStack (calc.tests.TestCalculator) ... ok
test_evalString (calc.tests.TestCalculator) ... ok
test_parseString (calc.tests.TestCalculator) ... ok
--------------------------------------------------------------
Ran 3 tests in 0.010s
OK
$ git add .
$ git commit -m "test evalStack, evalString, parseString: ok"
該提交程式碼與測試了
關於測試
我說的其實是......
單元測試也是程式碼的⼀一部分。
待辦清單
• 數字求值:解析字串、解析 exprStack
• 錯誤處理
• 處理簡單數學運算
• 處理先乘除後加減
• 處理括號運算
• 交換律、結合律、分配律
• 確認驗收測試通過
增加測試
from django.test import TestCase
from calc.calculator import Calculator
# Create your tests here.
class TestCalculator(TestCase):
def setUp(self):
self.calc = Calculator()
…(略)
def test_invalid_input(self):
evalString = self.calc.evalString
self.assertEqual(evalString('hello world'), 'Invalid
Input')
你無法避免使⽤用者出怪招
執⾏行測試
$ python manage.py test
==============================================================
ERROR: test_invalid_input (calc.tests.TestCalculator)
--------------------------------------------------------------
Traceback (most recent call last):
...(略)
File "/home/vagrant/myWorkspace/venv/lib/python3.5/site-
packages/pyparsing.py", line 1936, in parseImpl
raise ParseException(instring, loc, self.errmsg, self)
pyparsing.ParseException: Expected W:(0123...) (at char 0),
(line:1, col:1)
溫馨提⽰示:⺫⽬目前有看不懂的表⽰示式
進⾏行實作
class Calculator:
...(略)
def evalString(self, string):
try:
self.parseString(string)
return self.evalStack(self.exprStack)
except ParseException:
return 'Invalid Input'
針對無法解析的字串,預設回覆值
執⾏行測試
$ python manage.py test
....
--------------------------------------------------------------
Ran 4 tests in 0.005s
OK
$ git add .
$ git commit -m "handle ParseException"
該提交程式碼與測試了
關於測試
我說的其實是......
Test
Code Refactor
Start
待辦清單
• 數字求值:解析字串、解析 exprStack
• 錯誤處理
• 處理簡單數學運算:解析字串、字串求值
• 處理先乘除後加減
• 處理括號運算
• 交換律、結合律、分配律
• 確認驗收測試通過
增加測試
from django.test import TestCase
from calc.calculator import Calculator
# Create your tests here.
class TestCalculator(TestCase):
def setUp(self):
self.calc = Calculator()
…(略)
def test_parseString(self):
parseString = self.calc.parseString
self.assertEqual(parseString('0'), ['0'])
self.assertEqual(parseString('1'), ['1'])
self.assertEqual(parseString('3+2'), ['3', '2', '+'])
Baby step,再前進⼀一⼩小步
執⾏行測試
$ python manage.py test -v2
==============================================================
ERROR: test_parseString (calc.tests.TestCalculator)
--------------------------------------------------------------
Traceback (most recent call last):
File "/home/vagrant/myWorkspace/demo/calc/tests.py", line
14, in test_parseString
self.assertEqual(parseString('3+2'), ['3', '2', '+'])
...(略)
raise ParseException(instring, loc, self.errmsg, self)
pyparsing.ParseException: Expected end of text (at char 1),
(line:1, col:2)
溫馨提⽰示:⺫⽬目前有看不懂的表⽰示式
進⾏行實作
class Calculator:
def __init__(self):
self.exprStack = []
def pushStack(s, l, t):
self.exprStack.append(t[0])
integer = Word(nums).addParseAction(pushStack)
op = Literal('+') | Literal('-') | Literal('*') | Literal('/')
expr = integer + ZeroOrMore((op + integer).addParseAction(pushStack))
self.expr = expr + StringEnd()
def parseString(self, string):
self.exprStack = []
self.expr.parseString(string)
return self.exprStack
integer :: '0'...'9'*
op :: '+' | '-' | '*' | '/'
expr :: integer [op integer]*
執⾏行測試
$ python manage.py test -v2
...(略)
test_evalStack (calc.tests.TestCalculator) ... ok
test_evalString (calc.tests.TestCalculator) ... ok
test_invalid_input (calc.tests.TestCalculator) ... ok
test_parseString (calc.tests.TestCalculator) ... ok
--------------------------------------------------------------
Ran 4 tests in 0.007s
OK
$ git add .
$ git commit -m "parseString of '3+2': ok"
該提交程式碼與測試了
待辦清單
• 數字求值:解析字串、解析 exprStack
• 錯誤處理
• 處理簡單數學運算:解析字串、字串求值
• 處理先乘除後加減
• 處理括號運算
• 交換律、結合律、分配律
• 確認驗收測試通過
增加測試
from django.test import TestCase
from calc.calculator import Calculator
# Create your tests here.
class TestCalculator(TestCase):
def setUp(self):
self.calc = Calculator()
…(略)
def test_num_op_num(self):
evalString = self.calc.evalString
self.assertEqual(evalString('3+2'), 5)
self.assertEqual(evalString('3-2'), 1)
self.assertEqual(evalString('3*2'), 6)
self.assertEqual(evalString('3/2'), 1.5)
測試由算數表⽰示式求值
執⾏行測試
$ python manage.py test
==============================================================
ERROR: test_num_op_num (calc.tests.TestCalculator)
--------------------------------------------------------------
Traceback (most recent call last):
…(略)
File "/home/vagrant/myWorkspace/demo/calc/calculator.py",
line 29, in evalStack
return float(op)
ValueError: could not convert string to float: '+'
溫馨提⽰示:無法處理加號
進⾏行實作
from calc.scalc import SimpleCalculator
class Calculator:
def __init__(self):
…(略)
calc = SimpleCalculator()
self.opfun = {
'+' : (lambda a, b: calc.add(a,b)),
'-' : (lambda a, b: calc.sub(a,b)),
'*' : (lambda a, b: calc.mul(a,b)),
'/' : (lambda a, b: calc.div(a,b)) }
def evalStack(self, stack):
op = stack.pop()
if op in '+-*/':
op2 = self.evalStack(stack)
op1 = self.evalStack(stack)
return self.opfun[op](op1, op2)
else:
return float(op)
使⽤用另⼀一個團隊開發的模組
呼叫處理符號的⽅方法
對應符號與處理⽅方法
執⾏行測試
$ python manage.py test
==============================================================
ERROR: test_num_op_num (calc.tests.TestCalculator)
--------------------------------------------------------------
Traceback (most recent call last):
...(略)
File "/home/vagrant/myWorkspace/demo/calc/calculator.py",
line 25, in <lambda>
'+' : (lambda a, b: calc.add(a,b)),
File "/home/vagrant/myWorkspace/demo/calc/scalc.py", line 6,
in add
raise NotImplementedError
NotImplementedError
溫馨提⽰示:SimpleCalculator.add ⽅方法尚未實作
NotImplementedError?!
相依元件尚未完成
#file: calc/scalc.py
class SimpleCalculator:
def add(self, a, b):
raise NotImplementedError
def sub(self, a, b):
raise NotImplementedError
def mul(self, a, b):
raise NotImplementedError
def div(self, a, b):
raise NotImplementedError
圖⽚片來源 http://goo.gl/ciEspm
眼前有兩條路
都是 they 的錯 It’s time to
Facebook
⼩小兵 idol…
⾏行不⾏行啊
⾛走!買飲料沒事做,
趕快裝忙
等
使⽤用測試替⾝身 (Test Double)
圖⽚片來源 http://goo.gl/71wmBt
整合測試
SUT: System Under Test
DOC: Depended-on Component
Setup
Exercise
Verify
Teardown
SUT DOC
單元測試
SUT Test Double
Setup
Exercise
Verify
Teardown
測試替⾝身
• 種類
• Test spy, Test stub, Mock object, Fack object,
Dummy object
• ⺫⽬目的
• 不依賴其他元件,做到真正的單元測試
• 可控制的測試環境
待辦清單
• 數字求值:解析字串、解析 exprStack
• 錯誤處理
• 處理簡單數學運算:解析字串、字串求值
• 處理先乘除後加減
• 處理括號運算
• 交換律、結合律、分配律
• 確認驗收測試通過
Mock object
依賴注⼊入
(Dependency Injection, DI)
Builder Calculator
SimpleCalculator
interface
add(), sub(),
mul(), div()
3. use
1. create
2. inject independence
進⾏行實作
class Calculator:
def __init__(self, calc):
…(略)
calc = SimpleCalculator()
self.opfun = {
'+' : (lambda a, b: calc.add(a,b)),
'-' : (lambda a, b: calc.sub(a,b)),
'*' : (lambda a, b: calc.mul(a,b)),
'/' : (lambda a, b: calc.div(a,b)) }
依賴注⼊入
修改測試
class TestCalculator(TestCase):
def setUp(self):
add_dict = {(3,2) : 5}
sub_dict = {(3,2) : 1}
mul_dict = {(3,2) : 6}
div_dict = {(3,2) : 1.5}
def add(*args): return add_dict[args]
def sub(*args): return sub_dict[args]
def mul(*args): return mul_dict[args]
def div(*args): return div_dict[args]
scalc = SimpleCalculator()
scalc.add = MagicMock(side_effect = add)
scalc.sub = MagicMock(side_effect = sub)
scalc.mul = MagicMock(side_effect = mul)
scalc.div = MagicMock(side_effect = div)
self.calc = Calculator(scalc)
...(略)
測試替⾝身
依賴注⼊入
只會回答 3+2, 3-2, 3*2, 3/2 的問題
執⾏行測試
$ python manage.py test
.....
--------------------------------------------------------------
Ran 5 tests in 0.012s
OK
$ git add .
$ git commit -m "DI and mock of SimpleCalculator"
該提交程式碼與測試了
實作細節講太多
讓⼈人昏昏欲睡
接下來稍微加速⼀一下
圖⽚片來源 http://goo.gl/IpX6lw
待辦清單
• 數字求值:解析字串、解析 exprStack
• 錯誤處理
• 處理簡單數學運算:解析字串、字串求值
• 處理先乘除後加減
• 處理括號運算
• 交換律、結合律、分配律
• 確認驗收測試通過
Mock object
增加測試
class TestCalculator(TestCase):
def setUp(self):
add_dict = {…}
sub_dict = {…}
mul_dict = {…}
div_dict = {…}
…(略)
def test_order_of_operations(self):
evalString = self.calc.evalString
self.assertEqual(evalString('4+3*2'), 10)
self.assertEqual(evalString('9-3*2+2/1'), 5)
增加偽造的範圍
測試先乘除、後加減
執⾏行測試
$ python manage.py test
...(略)
==============================================================
FAIL: test_order_of_operations (calc.tests.TestCalculator)
--------------------------------------------------------------
Traceback (most recent call last):
File "/home/vagrant/myWorkspace/demo/calc/tests.py", line
62, in test_order_of_operations
self.assertEqual(evalString('4+3*2'), 10)
AssertionError: 14.0 != 10
溫馨提⽰示: 還沒處理“先乘除、後加減”
進⾏行實作
class Calculator:
def __init__(self, calc):
self.exprStack = []
def pushStack(s, l, t):
self.exprStack.append(t[0])
integer = Word(nums).addParseAction(pushStack)
addop = Literal('+') | Literal('-')
mulop = Literal('*') | Literal('/')
atom = integer
term = atom + ZeroOrMore((mulop + atom).addParseAction(pushStack))
expr = term + ZeroOrMore((addop + term).addParseAction(pushStack))
self.expr = expr + StringEnd()
...(略)
integer :: '0'...'9'*
addop :: '+' | '-'
mulop :: '*' | '/'
atom :: integer
term :: atom [mulop atom]*
expr :: term [addop term]*
執⾏行測試
$ python manage.py test
......
--------------------------------------------------------------
Ran 6 tests in 0.018s
OK
$ git add .
$ git commit -m "evalString can handle the order of
operations"
該提交程式碼與測試了
待辦清單
• 數字求值:解析字串、解析 exprStack
• 錯誤處理
• 處理簡單數學運算:解析字串、字串求值
• 處理先乘除後加減
• 處理括號運算
• 交換律、結合律、分配律
• 確認驗收測試通過
Mock object
增加測試
class TestCalculator(TestCase):
def setUp(self):
add_dict = {…}
sub_dict = {…}
mul_dict = {…}
div_dict = {…}
…(略)
def test_parentheses(self):
evalString = self.calc.evalString
self.assertEqual(evalString('(4+3)*2'), 14)
self.assertEqual(evalString('(9-3)*(2+2)/1'), 24)
增加偽造的範圍
測試括號運算
執⾏行測試
$ python manage.py test
...(略)
==============================================================
FAIL: test_parentheses (calc.tests.TestCalculator)
--------------------------------------------------------------
Traceback (most recent call last):
File "/home/vagrant/myWorkspace/demo/calc/tests.py", line
67, in test_parentheses
self.assertEqual(evalString('(4+3)*2'), 14)
AssertionError: 'Invalid Input' != 14
溫馨提⽰示: 還沒處理括號運算
進⾏行實作
class Calculator:
def __init__(self, calc):
...(略)
integer = Word(nums).addParseAction(pushStack)
addop = Literal('+') | Literal('-')
mulop = Literal('*') | Literal('/')
lpar = Literal('(')
rpar = Literal(')')
expr = Forward()
atom = integer | lpar + expr + rpar
term = atom + ZeroOrMore((mulop + atom).addParseAction(pushStack))
expr << term + ZeroOrMore((addop + term).addParseAction(pushStack))
self.expr = expr + StringEnd()
...(略)
integer :: '0'...'9'*
addop :: '+' | '-'
mulop :: '*' | '/'
atom :: integer | '(' + expr + ')'
term :: atom [mulop atom]*
expr :: term [addop term]*
執⾏行測試
$ python manage.py test
.......
--------------------------------------------------------------
Ran 7 tests in 0.015s
OK
$ git add .
$ git commit -m "evalString can handle parentheses"
該提交程式碼與測試了
待辦清單
• 數字求值:解析字串、解析 exprStack
• 錯誤處理
• 處理簡單數學運算:解析字串、字串求值
• 處理先乘除後加減
• 處理括號運算
• 交換律、結合律、分配律
• 確認驗收測試通過
Mock object
好消息!
SimpleCalculator 完成
相依元件已經完成
#file: calc/scalc.py
class SimpleCalculator:
def add(self, a, b):
return a+b
def sub(self, a, b):
return a-b
def mul(self, a, b):
return a*b
def div(self, a, b):
return a/b
待辦清單
• 數字求值:解析字串、解析 exprStack
• 錯誤處理
• 處理簡單數學運算:解析字串、字串求值
• 處理先乘除後加減
• 處理括號運算
• 交換律、結合律、分配律
• 確認驗收測試通過
Mock object
換上 SimpleCalculator
修改測試
class TestCalculator(TestCase):
def setUp(self):
"""
add_dict = {…}
sub_dict = {…}
mul_dict = {…}
div_dict = {…}
def add(*args): return add_dict[args]
def sub(*args): return sub_dict[args]
def mul(*args): return mul_dict[args]
def div(*args): return div_dict[args]
scalc = SimpleCalculator()
scalc.add = MagicMock(side_effect = add)
scalc.sub = MagicMock(side_effect = sub)
scalc.mul = MagicMock(side_effect = mul)
scalc.div = MagicMock(side_effect = div)
"""
self.calc = Calculator()
...(略)
註解程式碼
不傳 mock object 進去
修改實作
from calc.scalc import SimpleCalculator
...(略)
class Calculator:
def __init__(self, calc = SimpleCalculator()):
self.exprStack = []
def pushStack(s, l, t):
self.exprStack.append(t[0])
…(略)
預設使⽤用 SimpleCalculator
執⾏行測試
$ python manage.py test
.......
--------------------------------------------------------------
Ran 7 tests in 0.015s
OK
$ git add .
$ git commit -m "use SimpleCalculator instead of mock object"
該提交程式碼與測試了
待辦清單
• 數字求值:解析字串、解析 exprStack
• 錯誤處理
• 處理簡單數學運算:解析字串、字串求值
• 處理先乘除後加減
• 處理括號運算
• 交換律、結合律、分配律
• 確認驗收測試通過
Mock object
換上 SimpleCalculator
增加測試
class TestCalculator(TestCase):
…(略)
def test_commutative_property(self):
evalString = self.calc.evalString
self.assertEqual(evalString('3+4'), evalString('4+3'))
self.assertEqual(evalString('2*5'), evalString('5*2'))
def test_associative_property(self):
evalString = self.calc.evalString
self.assertEqual(evalString('(5+2) + 1'), evalString('5 + (2+1)'))
self.assertEqual(evalString('(5*2) * 3'), evalString('5 * (2*3)'))
def test_distributive_property(self):
evalString = self.calc.evalString
self.assertEqual(evalString('2 * (1+3)'), evalString('(2*1) + (2*3)'))
self.assertEqual(evalString('(1+3) * 2'), evalString('(1*2) + (3*2)'))
分配律測試
結合律測試
交換律測試
執⾏行測試
$ python manage.py test
..........
--------------------------------------------------------------
Ran 10 tests in 0.021s
OK
$ git add .
$ git commit -m "satisfy commutative, associative,
distributive properties"
該提交程式碼與測試了
關於測試
我說的其實是......
“I get paid for code that works, not
for tests, so my philosophy is to
test as little as possible to reach a
given level of confidence.”
- Kent Beck’s answer to How deep are your unit tests?
待辦清單
• 數字求值:解析字串、解析 exprStack
• 錯誤處理
• 處理簡單數學運算:解析字串、字串求值
• 處理先乘除後加減
• 處理括號運算
• 交換律、結合律、分配律
• 確認驗收測試通過
Mock object
換上 SimpleCalculator
執⾏行測試
$ python manage.py behave
Creating test database for alias 'default'...
Feature: Web calculator # features/calc.feature:3
As a student
In order to finish my homework
I want to do arithmatical operations
Scenario Outline: do simple operations -- @1.1
Given I enter 3 + 2
When I press "=" button
Then I get the answer 5
…(略)
1 feature passed, 0 failed, 0 skipped
12 scenarios passed, 0 failed, 0 skipped
36 steps passed, 0 failed, 0 skipped, 0 undefined
關於測試
我說的其實是......
各層測試的⺫⽬目不⼀一樣
User Story
Design Unit Tests
Acceptance
Tests
make sure you
do things right
make sure you
do right things
待辦清單
• 數字求值:解析字串、解析 exprStack
• 錯誤處理
• 處理簡單數學運算:解析字串、字串求值
• 處理先乘除後加減
• 處理括號運算
• 交換律、結合律、分配律
• 確認驗收測試通過
Mock object
換上 SimpleCalculator
測試覆蓋率
測試成功趨勢
靜態檢查結果
專案健康狀態
關於測試
我說的其實是......
讓⾃自動化測試
成為⼀一張開發的安全網
醜媳婦⾒見公婆
圖⽚片來源 http://goo.gl/yMDEQd
Templates
Views
Browser
Models
Database
URLs
Django MTV
Template
<form ation="." method="post">
{% csrf_token %}
<input id="expr" type="text" name="expr" value="{{ value }}">
<input type="submit" value="=">
</form>
View
from django.shortcuts import render
from django.http import HttpResponse
from .calculator import Calculator
# Create your views here.
def calc(request):
value = ''
if request.method == 'POST':
calc = Calculator()
expr = request.POST['expr']
value = calc.evalString(expr)
return render(request, 'calculator.html', {'value': value})
這層越薄越好
URL
urlpatterns = [
...(略)
url(r'^$', calc_views.calc),
]
關於測試
我說的其實是......
圖⽚片來源 http://goo.gl/Jgmgjy
透過 Demo
測試、回饋、學習
提交版本
$ git add .
$ git commit -m "add template/view/URL of web calculator"
還有哪些測試?
Automated
Functional Acceptance
Tests
Manual
Showcases
Usability Testing
Exploratory Testing
Unit Tests
Component Tests
System Tests
Automated
Nonfunctional
Acceptance Tests
(Capacity, Security,
Availability…)
Manual/Automated
Testing quadrant diagram
Supportingprogramming
Critiqueproject
Business facing
Technology facing
探索式測試
(Exploratory Testing)
• “所謂探索式測試, 就是同時進⾏行分析系統, 學習系
統, 設計測試, 執⾏行測試等動作. 因為⼀一開始對受測
系統不太懂, 無法開始就設計到位, 需要先執⾏行⼀一
下系統, 了解他是什麼, 同時也思考要如何規劃設
計. 因此, 這幾個動作是交錯在⼀一起進⾏行的. 這就像
敏捷開發⼀一樣, 設計, 開發和測試會同時發⽣生, 不應
該分成不同階段.” - David Ko
• 除以零會爆炸、不⽀支援⼩小數點、不⽀支援負數…
關於測試
我說的其實是......
唯有⾃自動化才能把⼈人⼒力
從瑣碎的⼿手動測試解放出來,
去做有價值的⼯工作。
Live Demo
DOC
Recap
User Stories
As a …, I want …, So that …
Scenarios
Given … When … Then …
Steps
@given(…)
def step_impl(context, …):
…
@when(…)
def step_impl(context, …):
…
@then(…)
def step_impl(context, …):
…
Interface
between the delegated tasks
and the domain model
Domain Models
Production code here…
Database Filesystem
Network
3rd party
library
Unit Tests
Test code here…
Test Double
Spy, Stub, Mock…
PM
PM
QA
RD
QA RD
Hardware
Remote
service
關於測試
我說的其實是......
只要有⼼心,
⼈人⼈人都可以玩測試。
學習筆記放在
https://github.com/hugolu/learn-test
Any question,
please send me pull requests :)

Más contenido relacionado

Similar a 關於測試,我說的其實是......

COSCUP: Introduction to Julia
COSCUP: Introduction to JuliaCOSCUP: Introduction to Julia
COSCUP: Introduction to Julia岳華 杜
 
Python testing using mock and pytest
Python testing using mock and pytestPython testing using mock and pytest
Python testing using mock and pytestSuraj Deshmukh
 
Tdd for BT E2E test community
Tdd for BT E2E test communityTdd for BT E2E test community
Tdd for BT E2E test communityKerry Buckley
 
Introduction to julia
Introduction to juliaIntroduction to julia
Introduction to julia岳華 杜
 
MT_01_unittest_python.pdf
MT_01_unittest_python.pdfMT_01_unittest_python.pdf
MT_01_unittest_python.pdfHans Jones
 
Numerical tour in the Python eco-system: Python, NumPy, scikit-learn
Numerical tour in the Python eco-system: Python, NumPy, scikit-learnNumerical tour in the Python eco-system: Python, NumPy, scikit-learn
Numerical tour in the Python eco-system: Python, NumPy, scikit-learnArnaud Joly
 
Behaviour Driven Development and Thinking About Testing
Behaviour Driven Development and Thinking About TestingBehaviour Driven Development and Thinking About Testing
Behaviour Driven Development and Thinking About Testingdn
 
Bdd and-testing
Bdd and-testingBdd and-testing
Bdd and-testingmalcolmt
 
Intel IPP Samples for Windows - error correction
Intel IPP Samples for Windows - error correctionIntel IPP Samples for Windows - error correction
Intel IPP Samples for Windows - error correctionAndrey Karpov
 
Intel IPP Samples for Windows - error correction
Intel IPP Samples for Windows - error correctionIntel IPP Samples for Windows - error correction
Intel IPP Samples for Windows - error correctionPVS-Studio
 
Parallel R in snow (english after 2nd slide)
Parallel R in snow (english after 2nd slide)Parallel R in snow (english after 2nd slide)
Parallel R in snow (english after 2nd slide)Cdiscount
 
Node.js System: The Landing
Node.js System: The LandingNode.js System: The Landing
Node.js System: The LandingHaci Murat Yaman
 
Ehsan parallel accelerator-dec2015
Ehsan parallel accelerator-dec2015Ehsan parallel accelerator-dec2015
Ehsan parallel accelerator-dec2015Christian Peel
 
Static code analysis: what? how? why?
Static code analysis: what? how? why?Static code analysis: what? how? why?
Static code analysis: what? how? why?Andrey Karpov
 
Test Driven Development With Python
Test Driven Development With PythonTest Driven Development With Python
Test Driven Development With PythonSiddhi
 

Similar a 關於測試,我說的其實是...... (20)

How to fake_properly
How to fake_properlyHow to fake_properly
How to fake_properly
 
COSCUP: Introduction to Julia
COSCUP: Introduction to JuliaCOSCUP: Introduction to Julia
COSCUP: Introduction to Julia
 
Python testing using mock and pytest
Python testing using mock and pytestPython testing using mock and pytest
Python testing using mock and pytest
 
Tdd for BT E2E test community
Tdd for BT E2E test communityTdd for BT E2E test community
Tdd for BT E2E test community
 
Celery
CeleryCelery
Celery
 
Introduction to julia
Introduction to juliaIntroduction to julia
Introduction to julia
 
Golang dot-testing-lite
Golang dot-testing-liteGolang dot-testing-lite
Golang dot-testing-lite
 
MT_01_unittest_python.pdf
MT_01_unittest_python.pdfMT_01_unittest_python.pdf
MT_01_unittest_python.pdf
 
Numerical tour in the Python eco-system: Python, NumPy, scikit-learn
Numerical tour in the Python eco-system: Python, NumPy, scikit-learnNumerical tour in the Python eco-system: Python, NumPy, scikit-learn
Numerical tour in the Python eco-system: Python, NumPy, scikit-learn
 
Behaviour Driven Development and Thinking About Testing
Behaviour Driven Development and Thinking About TestingBehaviour Driven Development and Thinking About Testing
Behaviour Driven Development and Thinking About Testing
 
Bdd and-testing
Bdd and-testingBdd and-testing
Bdd and-testing
 
Intel IPP Samples for Windows - error correction
Intel IPP Samples for Windows - error correctionIntel IPP Samples for Windows - error correction
Intel IPP Samples for Windows - error correction
 
Intel IPP Samples for Windows - error correction
Intel IPP Samples for Windows - error correctionIntel IPP Samples for Windows - error correction
Intel IPP Samples for Windows - error correction
 
Parallel R in snow (english after 2nd slide)
Parallel R in snow (english after 2nd slide)Parallel R in snow (english after 2nd slide)
Parallel R in snow (english after 2nd slide)
 
Node.js System: The Landing
Node.js System: The LandingNode.js System: The Landing
Node.js System: The Landing
 
Ehsan parallel accelerator-dec2015
Ehsan parallel accelerator-dec2015Ehsan parallel accelerator-dec2015
Ehsan parallel accelerator-dec2015
 
Test driven development
Test driven developmentTest driven development
Test driven development
 
Static code analysis: what? how? why?
Static code analysis: what? how? why?Static code analysis: what? how? why?
Static code analysis: what? how? why?
 
Xgboost
XgboostXgboost
Xgboost
 
Test Driven Development With Python
Test Driven Development With PythonTest Driven Development With Python
Test Driven Development With Python
 

Más de hugo lu

WSO2 IoTS Device Manufacturer Guide
WSO2 IoTS Device Manufacturer GuideWSO2 IoTS Device Manufacturer Guide
WSO2 IoTS Device Manufacturer Guidehugo lu
 
Dev ops 簡介
Dev ops 簡介Dev ops 簡介
Dev ops 簡介hugo lu
 
Sql injection 幼幼班
Sql injection 幼幼班Sql injection 幼幼班
Sql injection 幼幼班hugo lu
 
Sql or no sql, that is the question
Sql or no sql, that is the questionSql or no sql, that is the question
Sql or no sql, that is the questionhugo lu
 
Continuous integration
Continuous integrationContinuous integration
Continuous integrationhugo lu
 
Swift 2.0 的新玩意
Swift 2.0 的新玩意Swift 2.0 的新玩意
Swift 2.0 的新玩意hugo lu
 
精實執行工作坊
精實執行工作坊精實執行工作坊
精實執行工作坊hugo lu
 
Testing in swift
Testing in swiftTesting in swift
Testing in swifthugo lu
 
畫出商業模式
畫出商業模式畫出商業模式
畫出商業模式hugo lu
 
The linux networking architecture
The linux networking architectureThe linux networking architecture
The linux networking architecturehugo lu
 
精實軟體度量
精實軟體度量精實軟體度量
精實軟體度量hugo lu
 
看板實驗室
看板實驗室看板實驗室
看板實驗室hugo lu
 
嵌入式測試驅動開發
嵌入式測試驅動開發嵌入式測試驅動開發
嵌入式測試驅動開發hugo lu
 

Más de hugo lu (13)

WSO2 IoTS Device Manufacturer Guide
WSO2 IoTS Device Manufacturer GuideWSO2 IoTS Device Manufacturer Guide
WSO2 IoTS Device Manufacturer Guide
 
Dev ops 簡介
Dev ops 簡介Dev ops 簡介
Dev ops 簡介
 
Sql injection 幼幼班
Sql injection 幼幼班Sql injection 幼幼班
Sql injection 幼幼班
 
Sql or no sql, that is the question
Sql or no sql, that is the questionSql or no sql, that is the question
Sql or no sql, that is the question
 
Continuous integration
Continuous integrationContinuous integration
Continuous integration
 
Swift 2.0 的新玩意
Swift 2.0 的新玩意Swift 2.0 的新玩意
Swift 2.0 的新玩意
 
精實執行工作坊
精實執行工作坊精實執行工作坊
精實執行工作坊
 
Testing in swift
Testing in swiftTesting in swift
Testing in swift
 
畫出商業模式
畫出商業模式畫出商業模式
畫出商業模式
 
The linux networking architecture
The linux networking architectureThe linux networking architecture
The linux networking architecture
 
精實軟體度量
精實軟體度量精實軟體度量
精實軟體度量
 
看板實驗室
看板實驗室看板實驗室
看板實驗室
 
嵌入式測試驅動開發
嵌入式測試驅動開發嵌入式測試驅動開發
嵌入式測試驅動開發
 

Último

Virtual-Orientation-on-the-Administration-of-NATG12-NATG6-and-ELLNA.pdf
Virtual-Orientation-on-the-Administration-of-NATG12-NATG6-and-ELLNA.pdfVirtual-Orientation-on-the-Administration-of-NATG12-NATG6-and-ELLNA.pdf
Virtual-Orientation-on-the-Administration-of-NATG12-NATG6-and-ELLNA.pdfErwinPantujan2
 
Proudly South Africa powerpoint Thorisha.pptx
Proudly South Africa powerpoint Thorisha.pptxProudly South Africa powerpoint Thorisha.pptx
Proudly South Africa powerpoint Thorisha.pptxthorishapillay1
 
Judging the Relevance and worth of ideas part 2.pptx
Judging the Relevance  and worth of ideas part 2.pptxJudging the Relevance  and worth of ideas part 2.pptx
Judging the Relevance and worth of ideas part 2.pptxSherlyMaeNeri
 
ANG SEKTOR NG agrikultura.pptx QUARTER 4
ANG SEKTOR NG agrikultura.pptx QUARTER 4ANG SEKTOR NG agrikultura.pptx QUARTER 4
ANG SEKTOR NG agrikultura.pptx QUARTER 4MiaBumagat1
 
ISYU TUNGKOL SA SEKSWLADIDA (ISSUE ABOUT SEXUALITY
ISYU TUNGKOL SA SEKSWLADIDA (ISSUE ABOUT SEXUALITYISYU TUNGKOL SA SEKSWLADIDA (ISSUE ABOUT SEXUALITY
ISYU TUNGKOL SA SEKSWLADIDA (ISSUE ABOUT SEXUALITYKayeClaireEstoconing
 
FILIPINO PSYCHology sikolohiyang pilipino
FILIPINO PSYCHology sikolohiyang pilipinoFILIPINO PSYCHology sikolohiyang pilipino
FILIPINO PSYCHology sikolohiyang pilipinojohnmickonozaleda
 
Procuring digital preservation CAN be quick and painless with our new dynamic...
Procuring digital preservation CAN be quick and painless with our new dynamic...Procuring digital preservation CAN be quick and painless with our new dynamic...
Procuring digital preservation CAN be quick and painless with our new dynamic...Jisc
 
Keynote by Prof. Wurzer at Nordex about IP-design
Keynote by Prof. Wurzer at Nordex about IP-designKeynote by Prof. Wurzer at Nordex about IP-design
Keynote by Prof. Wurzer at Nordex about IP-designMIPLM
 
ENGLISH6-Q4-W3.pptxqurter our high choom
ENGLISH6-Q4-W3.pptxqurter our high choomENGLISH6-Q4-W3.pptxqurter our high choom
ENGLISH6-Q4-W3.pptxqurter our high choomnelietumpap1
 
INTRODUCTION TO CATHOLIC CHRISTOLOGY.pptx
INTRODUCTION TO CATHOLIC CHRISTOLOGY.pptxINTRODUCTION TO CATHOLIC CHRISTOLOGY.pptx
INTRODUCTION TO CATHOLIC CHRISTOLOGY.pptxHumphrey A Beña
 
THEORIES OF ORGANIZATION-PUBLIC ADMINISTRATION
THEORIES OF ORGANIZATION-PUBLIC ADMINISTRATIONTHEORIES OF ORGANIZATION-PUBLIC ADMINISTRATION
THEORIES OF ORGANIZATION-PUBLIC ADMINISTRATIONHumphrey A Beña
 
Incoming and Outgoing Shipments in 3 STEPS Using Odoo 17
Incoming and Outgoing Shipments in 3 STEPS Using Odoo 17Incoming and Outgoing Shipments in 3 STEPS Using Odoo 17
Incoming and Outgoing Shipments in 3 STEPS Using Odoo 17Celine George
 
Science 7 Quarter 4 Module 2: Natural Resources.pptx
Science 7 Quarter 4 Module 2: Natural Resources.pptxScience 7 Quarter 4 Module 2: Natural Resources.pptx
Science 7 Quarter 4 Module 2: Natural Resources.pptxMaryGraceBautista27
 
4.18.24 Movement Legacies, Reflection, and Review.pptx
4.18.24 Movement Legacies, Reflection, and Review.pptx4.18.24 Movement Legacies, Reflection, and Review.pptx
4.18.24 Movement Legacies, Reflection, and Review.pptxmary850239
 
Student Profile Sample - We help schools to connect the data they have, with ...
Student Profile Sample - We help schools to connect the data they have, with ...Student Profile Sample - We help schools to connect the data they have, with ...
Student Profile Sample - We help schools to connect the data they have, with ...Seán Kennedy
 
Visit to a blind student's school🧑‍🦯🧑‍🦯(community medicine)
Visit to a blind student's school🧑‍🦯🧑‍🦯(community medicine)Visit to a blind student's school🧑‍🦯🧑‍🦯(community medicine)
Visit to a blind student's school🧑‍🦯🧑‍🦯(community medicine)lakshayb543
 
ECONOMIC CONTEXT - PAPER 1 Q3: NEWSPAPERS.pptx
ECONOMIC CONTEXT - PAPER 1 Q3: NEWSPAPERS.pptxECONOMIC CONTEXT - PAPER 1 Q3: NEWSPAPERS.pptx
ECONOMIC CONTEXT - PAPER 1 Q3: NEWSPAPERS.pptxiammrhaywood
 

Último (20)

Virtual-Orientation-on-the-Administration-of-NATG12-NATG6-and-ELLNA.pdf
Virtual-Orientation-on-the-Administration-of-NATG12-NATG6-and-ELLNA.pdfVirtual-Orientation-on-the-Administration-of-NATG12-NATG6-and-ELLNA.pdf
Virtual-Orientation-on-the-Administration-of-NATG12-NATG6-and-ELLNA.pdf
 
Proudly South Africa powerpoint Thorisha.pptx
Proudly South Africa powerpoint Thorisha.pptxProudly South Africa powerpoint Thorisha.pptx
Proudly South Africa powerpoint Thorisha.pptx
 
Judging the Relevance and worth of ideas part 2.pptx
Judging the Relevance  and worth of ideas part 2.pptxJudging the Relevance  and worth of ideas part 2.pptx
Judging the Relevance and worth of ideas part 2.pptx
 
ANG SEKTOR NG agrikultura.pptx QUARTER 4
ANG SEKTOR NG agrikultura.pptx QUARTER 4ANG SEKTOR NG agrikultura.pptx QUARTER 4
ANG SEKTOR NG agrikultura.pptx QUARTER 4
 
ISYU TUNGKOL SA SEKSWLADIDA (ISSUE ABOUT SEXUALITY
ISYU TUNGKOL SA SEKSWLADIDA (ISSUE ABOUT SEXUALITYISYU TUNGKOL SA SEKSWLADIDA (ISSUE ABOUT SEXUALITY
ISYU TUNGKOL SA SEKSWLADIDA (ISSUE ABOUT SEXUALITY
 
FILIPINO PSYCHology sikolohiyang pilipino
FILIPINO PSYCHology sikolohiyang pilipinoFILIPINO PSYCHology sikolohiyang pilipino
FILIPINO PSYCHology sikolohiyang pilipino
 
Procuring digital preservation CAN be quick and painless with our new dynamic...
Procuring digital preservation CAN be quick and painless with our new dynamic...Procuring digital preservation CAN be quick and painless with our new dynamic...
Procuring digital preservation CAN be quick and painless with our new dynamic...
 
Keynote by Prof. Wurzer at Nordex about IP-design
Keynote by Prof. Wurzer at Nordex about IP-designKeynote by Prof. Wurzer at Nordex about IP-design
Keynote by Prof. Wurzer at Nordex about IP-design
 
ENGLISH6-Q4-W3.pptxqurter our high choom
ENGLISH6-Q4-W3.pptxqurter our high choomENGLISH6-Q4-W3.pptxqurter our high choom
ENGLISH6-Q4-W3.pptxqurter our high choom
 
Raw materials used in Herbal Cosmetics.pptx
Raw materials used in Herbal Cosmetics.pptxRaw materials used in Herbal Cosmetics.pptx
Raw materials used in Herbal Cosmetics.pptx
 
INTRODUCTION TO CATHOLIC CHRISTOLOGY.pptx
INTRODUCTION TO CATHOLIC CHRISTOLOGY.pptxINTRODUCTION TO CATHOLIC CHRISTOLOGY.pptx
INTRODUCTION TO CATHOLIC CHRISTOLOGY.pptx
 
THEORIES OF ORGANIZATION-PUBLIC ADMINISTRATION
THEORIES OF ORGANIZATION-PUBLIC ADMINISTRATIONTHEORIES OF ORGANIZATION-PUBLIC ADMINISTRATION
THEORIES OF ORGANIZATION-PUBLIC ADMINISTRATION
 
Incoming and Outgoing Shipments in 3 STEPS Using Odoo 17
Incoming and Outgoing Shipments in 3 STEPS Using Odoo 17Incoming and Outgoing Shipments in 3 STEPS Using Odoo 17
Incoming and Outgoing Shipments in 3 STEPS Using Odoo 17
 
Science 7 Quarter 4 Module 2: Natural Resources.pptx
Science 7 Quarter 4 Module 2: Natural Resources.pptxScience 7 Quarter 4 Module 2: Natural Resources.pptx
Science 7 Quarter 4 Module 2: Natural Resources.pptx
 
4.18.24 Movement Legacies, Reflection, and Review.pptx
4.18.24 Movement Legacies, Reflection, and Review.pptx4.18.24 Movement Legacies, Reflection, and Review.pptx
4.18.24 Movement Legacies, Reflection, and Review.pptx
 
Student Profile Sample - We help schools to connect the data they have, with ...
Student Profile Sample - We help schools to connect the data they have, with ...Student Profile Sample - We help schools to connect the data they have, with ...
Student Profile Sample - We help schools to connect the data they have, with ...
 
Model Call Girl in Tilak Nagar Delhi reach out to us at 🔝9953056974🔝
Model Call Girl in Tilak Nagar Delhi reach out to us at 🔝9953056974🔝Model Call Girl in Tilak Nagar Delhi reach out to us at 🔝9953056974🔝
Model Call Girl in Tilak Nagar Delhi reach out to us at 🔝9953056974🔝
 
YOUVE_GOT_EMAIL_PRELIMS_EL_DORADO_2024.pptx
YOUVE_GOT_EMAIL_PRELIMS_EL_DORADO_2024.pptxYOUVE_GOT_EMAIL_PRELIMS_EL_DORADO_2024.pptx
YOUVE_GOT_EMAIL_PRELIMS_EL_DORADO_2024.pptx
 
Visit to a blind student's school🧑‍🦯🧑‍🦯(community medicine)
Visit to a blind student's school🧑‍🦯🧑‍🦯(community medicine)Visit to a blind student's school🧑‍🦯🧑‍🦯(community medicine)
Visit to a blind student's school🧑‍🦯🧑‍🦯(community medicine)
 
ECONOMIC CONTEXT - PAPER 1 Q3: NEWSPAPERS.pptx
ECONOMIC CONTEXT - PAPER 1 Q3: NEWSPAPERS.pptxECONOMIC CONTEXT - PAPER 1 Q3: NEWSPAPERS.pptx
ECONOMIC CONTEXT - PAPER 1 Q3: NEWSPAPERS.pptx
 

關於測試,我說的其實是......