做pc端网站行情,网站顶部悬浮导航代码,网站建设一般好久到期,输入关键词进行搜索1、为甚需要进程池#xff0c;线程池 介绍官网#xff1a;https://docs.python.org/dev/library/concurrent.futures.htmlconcurrent.futures模块提供了高度封装的异步调用接口
ThreadPoolExecutor#xff1a;线程池#xff0c;提供异步调用
ProcessPoolExecutor: 进程池线程池 介绍官网https://docs.python.org/dev/library/concurrent.futures.htmlconcurrent.futures模块提供了高度封装的异步调用接口
ThreadPoolExecutor线程池提供异步调用
ProcessPoolExecutor: 进程池提供异步调用
Both implement the same interface, which is defined by the abstract Executor class. 2、基本方法 1、submit(fn, *args, **kwargs) 异步提交任务2、map(func, *iterables, timeoutNone, chunksize1) 取代for循环submit的操作3、shutdown(waitTrue)
相当于进程池的pool.close()pool.join()操作
waitTrue等待池内所有任务执行完毕回收完资源后才继续
waitFalse立即返回并不会等待池内的任务执行完毕
但不管wait参数为何值整个程序都会等到所有任务执行完毕
submit和map必须在shutdown之前4、result(timeoutNone) 取得结果5、add_done_callback(fn) 回调函数3、进程池 The ProcessPoolExecutor class is an Executor subclass that uses a pool of processes to execute calls asynchronously. ProcessPoolExecutor uses the multiprocessing module,
which allows it to side-step the Global Interpreter Lock but also means that only picklable objects can be executed and returned.class concurrent.futures.ProcessPoolExecutor(max_workersNone, mp_contextNone)
An Executor subclass that executes calls asynchronously using a pool of at most max_workers processes.
If max_workers is None or not given, it will default to the number of processors on the machine.If max_workers is lower or equal to 0, then a ValueError will be raised. from concurrent.futures import ProcessPoolExecutor, ThreadPoolExecutor
import os
import timedef task(name):print(%s is running 《pid: %s》 % (name, os.getpid()))time.sleep(2)if __name__ __main__:# p Process(targettask, args(子,))# p.start
pool ProcessPoolExecutor(4) # 进程池max_workers4个for i in range(10): # 总共执行10次每次4个进程的执行pool.submit(task, 子进程%s % i)print(主) 4、线程池 ThreadPoolExecutor is an Executor subclass that uses a pool of threads to execute calls asynchronously.
class concurrent.futures.ThreadPoolExecutor(max_workersNone, thread_name_prefix)
An Executor subclass that uses a pool of at most max_workers threads to execute calls asynchronously.Changed in version 3.5: If max_workers is None or not given,
it will default to the number of processors on the machine, multiplied by 5,
assuming that ThreadPoolExecutor is often used to overlap I/O instead of CPU work and the number of workers should be higher than the number of workers for ProcessPoolExecutor.New in version 3.6: The thread_name_prefix argument was added to allow users to control the threading.
Thread names for worker threads created by the pool for easier debugging. 5、map函数取代了forsubmit from concurrent.futures import ThreadPoolExecutor,ProcessPoolExecutorimport os,time,random
def task(n):print(%s is runing %os.getpid())time.sleep(random.randint(1,3))return n**2if __name__ __main__:executorThreadPoolExecutor(max_workers3)# for i in range(11):# futureexecutor.submit(task,i)
executor.map(task,range(1,12)) #map取代了forsubmit 6、异步调用与回调机制 1提交任务的两种方式 # 提交任务的两种方式
# 1、同步调用 提交完任务后拿到结果再执行下一行代码导致程序是串行执行
# 2、异步调用 提交完任务后不用等待任务执行完毕2同步调用 from concurrent.futures import ThreadPoolExecutor
import time
import random# 吃饭
def eat(name):print(%s is eat % name)time.sleep(random.randint(1,5))ret random.randint(7, 13) * #return {name: name, ret: ret}# 称重
def weight(body):name body[name]size len(body[ret])print(%s 现在的体重是%s %(name, size))if __name__ __main__:pool ThreadPoolExecutor(15)rice1 pool.submit(eat, alex).result() # 取得结果 # 执行函数eatweight(rice1) # 执行函数weight
rice2 pool.submit(eat, jack).result() weight(rice2)rice3 pool.submit(eat, tom).result() weight(rice3) 2同步调用2 3回调函数 4是钩子函数 钩子函数是Windows消息处理机制的一部分通过设置“钩子”应用程序可以在系统级对所有消息、事件进行过滤访问在正常情况下无法访问的消息。钩子的本质是一段用以处理系统消息的程序通过系统调用把它挂入系统 --- 百度百科的定义 对于前端来说钩子函数就是指再所有函数执行前我先执行了的函数即 钩住 我感兴趣的函数只要它执行我就先执行。此概念或者说现象跟AOP面向切面编程很像7.线程池爬虫应用 1requests模块 import requests# 输入网址得到网址的源代码response requests.get(http://www.cnblogs.com/venicid/p/8923096.html)
print(response) # 输出Response [200]
print(response.text) # 以文本格式输出 2线程池爬虫 import requests
import time
from concurrent.futures import ThreadPoolExecutor# 输入网址得到网址的源代码
def get_code(url):print(GET , url)response requests.get(url)time.sleep(3)code response.textreturn {url: url, code: code}# 打印源代码的长度
def print_len(ret):ret ret.result()url ret[url]code_len len(ret[code])print(%s length is %s % (url, code_len))if __name__ __main__:url_list [http://www.cnblogs.com/venicid/default.html?page2,http://www.cnblogs.com/venicid/p/8747383.html,http://www.cnblogs.com/venicid/p/8923096.html,]pool ThreadPoolExecutor(2)for i in url_list:pool.submit(get_code, i).add_done_callback(print_len)pool.map(get_code, url_list) 转载于:https://www.cnblogs.com/venicid/p/8923528.html
