Python Notes: Decorator

By definition, a decorator is a function that takes another function and extends the behavior of the latter function without explicitly modifying it. Built-in decorators such as @staticmethod, @classmethod, and @property work in the same way.

How decorator works? Let's take a look at the following example:

def my_decorator(some_func):
    def wrapper():
        some_func()
        print("Some function is being called.")

    return wrapper

def another_func():
    print("Another function is being called.")

foo = my_decorator(another_func)

foo()

You would see the following print on the screen:

"Another function is being called."
"Some function is being called."

As you can see, we pass some_func into a closure, and do something before or after calling this function without modifying its original behavior, and we return the function. As we already learned, python functions are just like other python objects, they are first class objects. The returned function could be called just as any other functions.

@decorator

The above example is already very similar to decorator. The difference is that decorator often comes with a @ symbol. This is an example of python syntax sugar, which often refers to syntax in a programming language that aims to make the things easy to read or to express. For example, the following is an example of a decorator:

@my_decorator
def another_func():
    print("Another function is being called.")

Decorator that takes any argument

In python, we can use *args and **kargs to represent arbitrary arguments. The following example shows how to take arbitrary arguments in a decorator:

def proxy(func):
    def wrapper(*args, **kargs):
        return func(*args, **kargs)
    return wrapper

Decorator with parameters

Sometimes we want the decorator to take parameters, for example, we should implement it in this way:

def decorator(argument):
    def real_decorator(function):
        def wrapper(*args, **kwargs):
            do_something_with_argument(argument)
            result = function(*args, **kwargs)
        return wrapper
    return real_decorator

Decorator tips

One practical tips when defining decorator is to use the functoolss.wraps, this function would keep all the meta data information of the original functions, including the function signature and docstring information.

import functools

def uppercase(func):
    @functools.wraps(func)
    def warpper():
        return func()
    return wrapper

Python Notes: function as first class object

Per history of python blog, everything in python are first class objects, that means all objects that could be named in the language (e.g., integers, strings, functions, classes, modules, methods, etc.) to have equal status. Th:at is, they can be assigned to variables, placed in lists, stored in dictionaries, passed as arguments, and so forth..

Essentially, functions return a value based on the given arguments. In Python, functions are first-class objects as well. This means that functions can be passed around, and used as arguments, just like any other value (e.g, string, int, float).

Internally, python use a common C data structure that are used everywhere in the interpreter to represent all objects, either it is a python function or a integer.

However, when it comes to python function as first class, there are subtle things to think about when doing design.

Think about the following function definition:

class A:
    def __init__(self, x):
        self.x = x

    def foo(self, bar):
        print self.x, bar

What would happen if you assign A.foo to a variable: b = A.foo. The first argument of the function would have to be the instance itself. To handle this problem, python 2 returns a unbound method, which is a warper around the original function, but it restrict that the first argument of the function has to be the object instance: a = A(), b(a). In python 3, however, this restriction is removed as the author found this is not very useful.

Let’s think about the second condition, when you have a instance of a class: a = A(1), b = a.foo. In this case, python would return a bound method which is a thin wrapper around the original function. Bound method stores the instance as a internal object and this object would be the default first argument when calling this function.

马尔萨斯和《人口原理》

马尔萨斯是十八世纪英国的人口学家和政治经济学家。他最被广为人知的作品是《人口论》,而其中又以马尔萨斯人口陷阱这一概念最为人所熟知。

马尔萨斯陷阱

马尔萨斯人口陷阱(Malthusian Trap)是说:

  • 在没有限制的情况下,人口会呈现指数型增长
  • 食物只会呈现现行增长
  • 人口的增长一定会超过食物增长,从而导致食物不足

马尔萨斯认为,人类有两种方式会避免或者延缓这一情况的出现:

  • 有意识的晚婚晚育。
  • 缩减人类寿命的时间,比如战争,瘟疫,饥荒等。

需要进一步阐明的事,马尔萨斯所说的极限,是说在系统达到均衡的条件下,处在社会最底层的那部分人永远只能生活在勉强温饱的生存线上。至于处于社会底层的人口数量有多大,这是随着社会制度不同而不同的。

马尔萨斯为什么会认为人口数量会有极限呢?因为存在着生产的极限,在极限附近,生产投入的回报是递减的,也就是说,随着投入的增加,单位产出实际上是减少的。为什么我们没有陷入马尔萨斯所说的陷阱当中呢?因为技术的进步,我们不断发现更新更有效的生产方法,使得生产的边际回报是递增的。另一方面,主要是因为现代发达国家的生育水平逐渐下降,有些国家甚至到了低于代际更替水平一下,带来实际人口的减少。这主要是城市化造成的,城市化带来和生育孩子机会成本的提高:妇女的劳动价值越来越高,为了生育小孩儿放弃的成本也就会越来越高。另一方面是,多生孩子意味着需要稀释在每个孩子身上的投入,从而可能降低生育质量。

工资铁律

从马尔萨斯定律推论出来的一个理论是工资铁律。工资铁律是说,对于非技术工人,实物工资在长期将永远倾向于接近仅足够维持工人最低生活所需的水平。工资铁律描述的启示是马尔萨斯的模型出于边界时的情形,如果工资继续降低,工人将无法维持生活,工人数量将停止增加,如果工资升高,工人数量将会上升,压低劳动成本,从而使得工资回到最低水平。但是我们并未看到工资铁律的实际出现,这是因为,新的技术,投资导致市场对于劳动力需求的增大,避免了因为劳工过剩而带来的工资降低。

马尔萨斯弹簧

马尔萨斯理论的另一个衍生是马尔萨斯弹簧。马尔萨斯弹簧模型是用于解释文明诞生所需要的剩余如何发生。我们都知道,因为有这些剩余,一个社会才会有能力供养不进行实际劳作的人口,从而使他们有机会发展科技,文学,艺术等等。但是为什么会产生这些剩余呢?常见的误区是认为这是由于技术带来的生产力的提高,导致实际产出多于供养人口所需要的实物数量,于是便产生了剩余。但是根据马尔萨斯的理论,人口数量总是会趋近于极限水平,也就是剩余为零的水平,这样的人口增长只需要几代人便可以实现。但是为何这种情况没有发生呢?一个重要原因就是税赋,因为税赋的存在,劳动人群的产出被拿走一部分,使得实际的粮食产出降低,人口也就不会增加到极限水平。这种因为税赋带来的对于人口的抑制作用便称为马尔萨斯弹簧。

不得不说这种解释是包含着很多洞见的,如果从税赋的最终效果来看,它们确实被用来供养一个广大的上层阶级,从而促进了社会的发展。但是另一方面,实际非劳动人口之所以可以生存下来并将它们的技艺发展提高,也是因为他们的产品实际上是被需要的。比如手工艺人制作的农具,比如喜剧演员的表演,都是劳动人群愿意购买的产品和服务,是市场价值,而不是强制的税赋供养了这些非劳动人口。单着并不意味着马尔萨斯弹簧的实效,实际上,马尔萨斯的理论强调的长期的趋势。马尔萨斯认为,从长期看,人们的生活水平总是因为实际人口的增加而降低的,而生活水平的降低意味着购买能力的降低,供养非劳动人口的市场缩小,文明的发展因此会被抑制。因此可以说税赋确实是长期一直人口增长的有效弹簧。

马尔萨斯与进化论

马尔萨斯的理论也启发了达尔文发明生物进化理论。生物进化理论要论证的是生物是通过不断在适应环境的过程中进化而来,而这是结果,生物进化的动因在于生存压力,其中最重要的压力之一是食物资源的压力,因为没有足够的食物资源,生物群体内部同样要发生激烈竞争。

马尔萨斯在起人口论的第十八章中阐明了人口压力对于文明发展的推动作用。马尔萨斯认为,正因为人口永远会朝向极限的状态发展,资源永远都是不足的,因此富有才智的人们才会愿意去努力发展自己。如果谁也不想在社会的阶梯上往上爬,谁也不担心会从社会的阶梯上摔落下来,如果勤劳得不到奖励,懒惰得不到惩罚,中产阶级就肯定不是现在这种样子了。 了解到奋斗这个过程的无可避免,或许是我们在奋斗路上的一个小小安慰。

Python Notes: Iterator, Generator and Co-routine

Iteration

Python support iteration, for example, iterating over a list:

for elem in [1, 2, 3]:
    print elem

Iterating over a dict:

for key in {'Google': 'G',
              'Yahoo': 'Y',
              'Microsoft': 'M'}:
    print key

Iterating over a file:

with open('path/to/file.csv', 'r') as file:
    for line in file:
        print line

We use iterable objects in many ways, for example, reductions: sum(s), min(s), constructors: list(s), in operators: item in s.

The reason why we can iterate over iterable is because of iterable protocols: any objects that supports iter() and next() is an itterable. For example, we can define one itterable object in the following way:

class count:

def __init__(self, start):
    self.count = start

def __iter__(self):
    return self

Def next(self):
    if self.count < 0:
        raise StopIteration
    r = self.count
    self.count -=1
    return r

We can use the above example in this way:

c = count(5)
for i in c:
    print I
# 5, 4, 3, 2, 1

Generator

So what is a generator? By definition: a generator is a function that produces a sequence of results instead of a single value.

So generator is a function, it is different from other functions that it generates a sequence of results instead of a single value. Generator function is very different from normal function, calling the generator function will create one generator, but would not execute it, until next() is called. The following is an example of generator:

def count(n):
    while n > 0:
        yield n
        n -= 1

c = count(5)

Note that when we first initiate count, it won't execute. Until the first time we call, c.next() the generator would start to execute. But it will suspend on the yield command, until next time it executes.

So to speak, a generator is a convenient way of writing an iterator, and you don't have to worry about iterator protocols.

Except for yield based generator function, python also supports generator expression:

a = [1, 2, 3, 4]
b = [x*2 for x in a]
c = (x*2 for x in a)

b is still a regular list, while c is a generator.

Co-routine

Python coroutine is very similar to generator. Think about the following pattern:

def receive_count():
    try:
        while True:
            n = (yield) # Yield expression
            print "T-minues ", n
    except GeneratorExit:
        print "Exit from generator."

The above form of generator is called coroutine. Coroutine is different from generator in that it receives data instead of generates data. Think of it as a consumer or receiver.

To use python co-routine, you need to call next() first so that the function executes to the yield field part, then you can use send to send the value to the function. For example:

    c = receive_count()
    c.next() # trigger to yield function
    c.send(1) # sending 1 to the co-routine.
    # prints "T-minus 1"

Python provided a decorator called @consumer to execute the next() function part. With the consumer decorator, the co-routine can be used directly.

Then the question is: why don't we just declare co-routine as a regular function where you can send the value to it directly instead of relying on the yield expression? Using coroutine in the given examples doesn't fully justify it's value. More often, people use co-routine to implement a application level multiple threading. I will introduce more about this later.

Python Notes: Context management

Python supports context management. Which often used when handling resources, for example, file, network connection. With statement helps make sure the resources are cleaned up or released. There are two major functions for context management: __enter__ and __exit__.

__enter__ is triggered when the with statement is first triggered, while __exit__ statement is triggered when the statement finishes execution.

One very common usage of with statement is when we open up files:

    with open('/path/to/file', 'r') as file:
        for line in file():
            print(line)

The with statement on this example will automatically close the file descriptor no matter how this with block exits.

Python with statement also supports nesting, for example:

    with open('/open/to/file', 'r') as infile:
        with open('write/to/file', 'w') as outfile:
            for line in infile:
                if line.startswith('Test'):
                    outfile.write(line)

If you want your code to support with statement based context management, just override the two functions on your code, for example:

class Resource:
    def __init__(self, res):
        self.res = res

    def __enter__(self):
        #define your code here

    def __exit__(self, exc_type, exc_value, traceback):
        #define your code here

There is another way to support context management, which is to use the contextlib.contextmanager, we will introduce this later.

 

Python Notes: Closure

 

The following is an example of python closure.

def outer_function(outter_arg):
    closure_var = outter_arg
    def inner_function(inner_arg):
        print("{} {}".format(closure_var, inner_arg))

    return inner_function

# Usage of a python closure
closure_func = outer_function("X")
closure_func("Y1") # print "Y1 X"
closure_funct("Y2") # output "Y2 X"

Variables and nested function

Python has two types of variables: local variable and global variable. Variables defined within a function has a local scope, while variables defined outside a function has global scope.

When a function is defined within another function, the function is called nested function. The nested function, however, can access the outer function’s variables. In the example above, outer function defined a variable called closure_var, and this variable would be initiated by the input variable. The inner function is able to access this variable and use this variable in its own function definition.

There are two steps of using a closure function: initiate the closure function and assign to a local variable, then call the variable with parameter to invoke the inner function.

When to use closure?

  • Reduce the use of global variables

Closure could hide variables inside the function, in this way we reduce the use of global variables.

  • Simplify single function classes.
    For example, the above example could be converted into a single function class:
class OuterClass:
    def __init__(outer_arg):
        self.arg = outer_arg

    def inner_function(self, inner_arg):
        print("{} {}".format(self.arg, inner_arg))

In general, closure runs faster than instance function calls.

《人类文化的起源》书评

为什么不同国家不同地区的文化相差如此之大?为什么当代的文化与古代甚至近代的文化这么不同?文化的演进是完全随机的吗?还是说存在着某种共同的推动力量?

马文-哈里斯试图在《人类文化的起源》一书中回答这个问题。他认为,人口压力带来的生产强化是解释这个问题的关键。在有效的避孕手段发明以前,人口的自然增长是难以得到有效控制的,为了应对人口增加带来的资源相对稀释,人类必须强化生产从而获得更多的产出:或者使用高有效的生产技术,或者提高资源的利用效率。但是生产的强化总是伴随着新一轮的人口增长,从而迫使人们进入下一轮的生产强化之中。作者在本书中用这把钥匙解释了诸多文化的现象。

作者在开篇先试图澄清了人们对原始社会的普遍误解。在一般的想象当中,原始社会的人们生活水平极差,安全和营养得不到保证,人们花费大量时间获得食物以填饱肚子。但是作者列举了研究的证据表明,在狩猎采集社会的人类实际上可以获得充足的营养。他们可以轻松捕获大型动物,并通过各种手段抑制人口增长,从而保证整个族群的生活会平不会降低。

那么为何人类会进入农业社会呢?作者认为主要是因为大型野兽因为人类的捕杀以及冰川时期的结束而渐渐消失,狩猎采集的生产效率逐渐降低,人们不得不转向更加有效地生产方式,那就是种植农业。但是,不同大陆在农业上的禀赋完全不同,比如有些大陆有更多可供驯化的动物和植物,从而是这些地区的生产相对容易。但另一些地区,比如中美洲,就没有可供驯化的野兽。这也为后来比如吃人习俗的出现埋下了种子。

作者接着用这一理论解释了战争的起源。在作者开来,战争的主要作用在于对人口的控制。战争将人口聚散,拉开了居民之间的距离,降低了整个地区的人口密度。而男性更高的战争价值使人们更愿意培养男婴,杀害女婴。进一步的,作者讨论了男权社会的形成原因。基本来说,作者认为是战争导致了大男子主义,进而导致了父亲社会,随父共居,新娘嫁妆这些丰富,作为对男性尚武的性奖励的一夫多妻制度,以及妇女较低的社会地位。作者还讨论了为什么很多女权主义者将母系社会作为女权合理化的证据,认为这是对历史社会形态的误读。作者还讨论使用弗洛伊德理论解释战争起源的错误之处。弗洛伊德的理论讲战争视作人类内在本能冲突的表现,这种解释缺乏解释力的地方在于其无法阐释战争的诸多形态,以及不同国家从最好战到最不好战的迅速变化。

另一个重要的文化现象是国家的起源。作者认为国家最初的形成是依赖于负责在分工的大人物。在最开始的时候,这些大人物仅仅是负责举办盛大的宴席,负责食物的再次分配。农业生产的强化导致了对协作的需求,从而加强了这些人的权力。这种权利的逐渐集中最后带来了原始国家的出现。作者在随后的章节中重点讨论了不同地区在国家产生之后完全不同的形态。

作者重点讨论了阿兹特克人食人传统的出现。作者认为,阿兹特克人的食人习俗是作为一种蛋白质再分配的方式存在的。在中美洲地区,因为缺乏动物资源,通过食人来获得更多的蛋白质是一种更加有效地途径。作者随后又讨论了基督徒祭神方式的转变。最开始的羔羊祭祀实际上也是一种蛋白质的再分配,后来资源的消耗以及人口的增长使得这种祭祀方式无法继续进行了。与阿兹特克人相比,这些地方之所以没有出现食人的习俗乃是因为这里存在着成本更低的获取蛋白质的方式。简单来说,是成本效益的对比决定了食人传统的出现与消失。另一个非常显著的现象是不同文化中食用动物的禁忌。比如伊斯兰教中猪是被禁止使用的,作者认为这是因为猪在地中海地区的饲养成本太高而导致的。这一地区缺乏饲养猪的植物以及猪降低体温需要的湿润土壤。而在印度,牛是一种被禁止使用的动物。这主要是因为牛是印度最重要的耕作动物,印度需要牛来维持农业生产。而这一文化之所以没有在中国出现,乃是因为中国具有更多的牧畜可以替代这些作用。

作者还讨论了水利国家和资本主义的现象。简单来说,在大河的滋润下贫瘠或者半贫瘠的平原河谷发展出来的社会容易形成水利社会。水利地区需要通过大规模协作修建水利工程,从而增加生产,这种对于大规模协作的需要是导致集权制度在这些地方反复出现的原因。而被迫强化生产以应付人口增长则是最根本的驱动力。与之对比,欧洲之所以没有被封建主义仿佛钳制乃是因为生产强化破势人们转变生产方式,而这种生态产方式更加青睐资本主义的制度而非封建主义制度。

作者在最后一章谈到了当代工业社会。作者认为,之所以我们暂时没有陷入生产强化资源耗尽的陷阱之中,乃是因为技术变革。最根本的是避孕方法的变革和职业的变革。避孕套的发明使人们找到了低成本的组织人口增长的方法。而家庭从生产功能中解放出来降低了人们继续生育的动力。因为生育子女的成本极大提高,子女未来能否获得好的经济回报的不确定性增加,而人们不再依赖于子女为家庭生产增加人手。新的廉价的能源,以及低于生殖潜力的人口增长速度暂时缓解了资源分散的压力,从而是当代人类获得了前所未有的生活水平的提高。但是作者也提醒我们不要过于乐观,最终这种方式能否持续仍然取决于人类能否找到替代能源,从而是人类从资源枯竭的宿命中拯救出来。

总的来说,本书的作者使用了非常理性的,甚至带有部分决定论的理论来解释了最重要的人类文化的几个方面。这种解释提供了相当多的新颖的理解世界的角度。比如在最后一章,作者提到,养老保险和医疗保险实际上正在代替原来养育子女中足以重要的一环:孩子对父母的反哺。但同时,作者在解释部分理论的时候仍然是不够清晰的。比如对战争的起源问题,通过战争达到控制人口的目的对于战争的解释是不完全令人信服的。