• 设为首页
  • 点击收藏
  • 手机版
    手机扫一扫访问
    迪恩网络手机版
  • 关注官方公众号
    微信扫一扫关注
    迪恩网络公众号

Python multiprocessing.Semaphore类代码示例

原作者: [db:作者] 来自: [db:来源] 收藏 邀请

本文整理汇总了Python中multiprocessing.Semaphore的典型用法代码示例。如果您正苦于以下问题:Python Semaphore类的具体用法?Python Semaphore怎么用?Python Semaphore使用的例子?那么恭喜您, 这里精选的类代码示例或许可以为您提供帮助。



在下文中一共展示了Semaphore类的20个代码示例,这些例子默认根据受欢迎程度排序。您可以为喜欢或者感觉有用的代码点赞,您的评价将有助于我们的系统推荐出更棒的Python代码示例。

示例1: call_START

    def call_START(self, mock_pid_ptid_mapping=MagicMock(has_key=lambda *args, **kwargs: False)):
        self.Locks = [Lock() for _ in range(0, 3)]
        self.NL_L = Semaphore(2)
        self.free_threads = Semaphore(5)
        mock_proc_inter_instance = MagicMock(free_threads=self.free_threads,
                                             Locks=self.Locks,
                                             NL_L=self.NL_L,
                                             pid_ptid_mapping=MagicMock(
                                                 has_key=lambda *args, **kwargs: False)
                                             )

        with patch("processing_interface.ProcessingInterface.Instance", side_effect=lambda *args, **kwargs: mock_proc_inter_instance):
            LP = LongProcess.LongProcess(self.test_dicoms_dirs,
                                         [os.path.split(dicom_dir)[-1] for dicom_dir in self.test_dicoms_dirs])

        self.__test_Locks()

        LP._LongProcess__START()

        self.__test_Locks()

        print LP._Process__success
        assert(LP._Process__success == "SUCCESS")
        assert(LP._Process__state == STATES.LongFS1)

        return LP
开发者ID:wonkykludge,项目名称:fsspmnl_webinterface,代码行数:26,代码来源:test_LongFS3.py


示例2: run

    def run(self, tasks, build_config, parallel_threads):
        semaphore = Semaphore(parallel_threads)
        process_finished_notify = Condition(Lock())
        while tasks.count_buildable_tasks() > 0:
            task = tasks.get_next()

            if task is None:
                self.wait_tasks_to_complete(parallel_threads, process_finished_notify, semaphore)
                continue

            semaphore.acquire()
            task.state = Task.State.RUNNING
            logging.debug("Starting task %s", task.name)
            self.start_new_process(process_finished_notify, semaphore, self.process_job, task, build_config)

        self.wait_tasks_to_complete(parallel_threads, process_finished_notify, semaphore)

        if tasks.count(Task.State.FAILED) > 0:
            logging.error('Some packages failed to build.')
            logging.error("  %s", tasks.print_name(Task.State.FAILED))
            return 1
        if tasks.count(Task.State.RUNNING) > 0:
            logging.error('Something went wrong, there are still some running tasks.')
            return 1
        if tasks.count(Task.State.NEW) > 0:
            logging.error('Something went wrong, there are still unprocessed tasks.')
            return 1

        logging.info("Build completed successfully.")
        return 0
开发者ID:project-ncl,项目名称:pnc-cli,代码行数:30,代码来源:tasks.py


示例3: block_until_processed

def block_until_processed(cookie_jar: CookieJar, cookie_paths: Sequence[str],
                          expected_number_of_calls_to_mark_as_complete: int):
    """
    Puts the given cookies into the cookie jar and wait until they have been completed/marked for reprocessing.
    :param cookie_jar: the cookie jar to put cookies to process into
    :param cookie_paths: the cookie paths to process
    :param expected_number_of_calls_to_mark_as_complete: the number of calls expected to the Cookie jar's
    `mark_as_complete` method
    """
    if cookie_jar.queue_length() != 0:
        raise RuntimeError("Already cookies in the jar")

    mark_as_complete_semaphore = Semaphore(0)
    original_mark_as_complete = cookie_jar.mark_as_complete

    def mark_as_complete(path: str):
        mark_as_complete_semaphore.release()
        original_mark_as_complete(path)

    cookie_jar.mark_as_complete = MagicMock(side_effect=mark_as_complete)

    for cookie_path in cookie_paths:
        cookie_jar.mark_for_processing(cookie_path)

    calls_to_mark_as_complete = 0
    while calls_to_mark_as_complete != expected_number_of_calls_to_mark_as_complete:
        mark_as_complete_semaphore.acquire()
        assert cookie_jar.mark_as_complete.call_count <= expected_number_of_calls_to_mark_as_complete
        calls_to_mark_as_complete += 1

    assert calls_to_mark_as_complete == cookie_jar.mark_as_complete.call_count
开发者ID:wtsi-hgi,项目名称:cookie-monster,代码行数:31,代码来源:_helpers.py


示例4: __init__

 def __init__(self, n, timeout=None):
     self.n = n
     self.to = timeout
     self.count = Value('i', 0)
     self.mutex = Semaphore(1)
     self.turnstile1 = Semaphore(0)
     self.turnstile2 = Semaphore(1)
开发者ID:Exteris,项目名称:spack,代码行数:7,代码来源:multiproc.py


示例5: launch_workers

def launch_workers(outfile, start_index, end_index, score_flag, force, verbose):
	BASE_URL = "http://www.ign.com/games/all-ajax?startIndex="
	
	
	# Synchronization Tools
	num_workers = Semaphore(MAX_NUM_PROCESSES)
	outfile_lock = Lock()
	urlopen_lock = Lock()
	stderr_lock = Lock()
	print_lock = Lock()
	
	# Write the categories
	if (outfile != None):
		outfile.write("title,link,platform,publisher,score,date\n")

	# Launch the workers
	processes = []
	curr_index = start_index;
	while curr_index <= end_index:
		curr_url = BASE_URL + str(curr_index)
	 	worker = Process(target=open_url_and_parse,
	 		args=(outfile, curr_url, score_flag, force, verbose,
	 			outfile_lock, urlopen_lock, stderr_lock, print_lock,
	 			num_workers))
	 	processes.append(worker)
	 	if verbose:
			print_lock.acquire()
			print "Launching worker for url: %s" % curr_url
			print_lock.release()
	 	num_workers.acquire()
	 	worker.start()
	 	curr_index += INDEX_INCREMENT; 
	for p in processes:
	 	p.join()
开发者ID:akshayka,项目名称:video-games,代码行数:34,代码来源:IGN_scraper.py


示例6: setCurrentSimulationTime

 def setCurrentSimulationTime(self, currentSimulationTime):
    
     semaphore = Semaphore()
     semaphore.acquire()
     self.__currentSimulationTime = currentSimulationTime
     semaphore.release()
     return self.__currentSimulationTime
开发者ID:krohan100,项目名称:pydssim,代码行数:7,代码来源:abstract_simulation.py


示例7: getPeerID

 def getPeerID(self, peerId):
     
     semaphore = Semaphore()
     semaphore.acquire()
     peer = self.__layout[peerId]
     semaphore.release()
     return peer
开发者ID:krohan100,项目名称:pydssim,代码行数:7,代码来源:abstract_network.py


示例8: run

def run():
    algo = parameters["algo"]
    files = [open(x) for x in parameters["files"]]
    configs = []
    p = parameters["params"]
    max_processes = 3
    semaphore = Semaphore(max_processes)

    # generate configurations as compination of possible
    # keys and product of values
    for keys in it.combinations(p.keys(), len(p.keys())):
        v = [p[k] for k in keys]
        for values in it.product(*v):
            config = {}
            for i, k in enumerate(keys):
                config[k] = values[i]
            configs.append(config)
    for f in files:
        for conf in configs:
            config = {"FILENAME": f.name}
            config.update(conf)

            f.seek(0)
            num_vars, clauses = parser.parse(f)

            p = MyProcess(target=run_algorithm, args=(algo, num_vars, clauses, config, semaphore))

            semaphore.acquire()
            p.start()
开发者ID:domoritz,项目名称:SoSAT,代码行数:29,代码来源:benchmark.py


示例9: getNeighbors

 def getNeighbors(self, peer):
             
     semaphore = Semaphore()
     semaphore.acquire()
     neighbors = self.__layout[peer.getId()].getNeighbors()
     semaphore.release()
     return neighbors
开发者ID:krohan100,项目名称:pydssim,代码行数:7,代码来源:topology.py


示例10: countNeighbors

 def countNeighbors(self, peer):
                    
     semaphore = Semaphore()
     semaphore.acquire()
     
     count = peer.countNeighbors()
     semaphore.release()
     return count
开发者ID:krohan100,项目名称:pydssim,代码行数:8,代码来源:topology.py


示例11: getNeighborIt

 def getNeighborIt(self, peer):
             
     semaphore = Semaphore()
     semaphore.acquire()
     neighbors = []
     for neighbor in self.__layout[peer.getId()].getNeighbors():
         neighbors.append(neighbor.getTargetPeer())
     neighborIt = neighbors.__iter__()
     semaphore.release()
     return neighborIt
开发者ID:krohan100,项目名称:pydssim,代码行数:10,代码来源:topology.py


示例12: addPeer

 def addPeer(self, peer):
     
     if self.__layout.has_key(peer.getPID()):
         return False
     
     semaphore = Semaphore()
     semaphore.acquire()
     
     self.__layout[peer.getPID()] = peer
     semaphore.release()
     NetworkLogger().resgiterLoggingInfo("Add peer %s in Layout Network "%(peer.getPID()))
     return self.__layout.has_key(peer.getPID())
开发者ID:krohan100,项目名称:pydssim,代码行数:12,代码来源:abstract_network.py


示例13: Msg

class Msg(object):
    """
    Data structure encapsulating a message.
    """

    def __init__(self, size):
        self.s_e = Semaphore(1)
        self.s_f = Semaphore(0)
        self.s_buf = Array(ct.c_ubyte, size)

    def send(self, func):
        self.s_e.acquire()
        self.s_buf.acquire()
        send_result = func(self.s_buf._obj)
        self.s_buf.release()
        self.s_f.release()
        return send_result

    def recv(self, func):
        self.s_f.acquire()
        self.s_buf.acquire()
        recv_result = func(self.s_buf._obj)
        self.s_buf.release()
        self.s_e.release()
        return recv_result
开发者ID:huhoo,项目名称:neon,代码行数:25,代码来源:image.py


示例14: ForkingWorker

class ForkingWorker(BaseWorker):

    def __init__(self, num_processes=1):
        # Set up sync primitives, to communicate with the spawned children
        self._semaphore = Semaphore(num_processes)
        self._slots = Array('i', [0] * num_processes)

    def spawn_child(self):
        """Forks and executes the job."""
        self._semaphore.acquire()    # responsible for the blocking

        # Select an empty slot from self._slots (the first 0 value is picked)
        # The implementation guarantees there will always be at least one empty slot
        for slot, value in enumerate(self._slots):
            if value == 0:
                break

        # The usual hardcore forking action
        child_pid = os.fork()
        if child_pid == 0:
            # Within child

            # Disable signal handlers
            signal.signal(signal.SIGINT, signal.SIG_IGN)
            signal.signal(signal.SIGTERM, signal.SIG_IGN)

            random.seed()
            try:
                self.fake_work()
            finally:
                # This is the new stuff.  Remember, we're in the child process
                # currently. When all work is done here, free up the current
                # slot (by writing a 0 in the slot position).  This
                # communicates to the parent that the current child has died
                # (so can safely be forgotten about).
                self._slots[slot] = 0
                self._semaphore.release()
                os._exit(0)
        else:
            # Within parent, keep track of the new child by writing its PID
            # into the first free slot index.
            self._slots[slot] = child_pid

    def wait_for_children(self):
        for child_pid in self._slots:
            if child_pid != 0:
                os.waitpid(child_pid, 0)

    def get_id(self):
        return os.getpid()
开发者ID:nvie,项目名称:worker-experiment,代码行数:50,代码来源:forking.py


示例15: disconnect

 def disconnect(self, priority):
     sem = Semaphore()
     sem.acquire()
     if not self.getPeer().isConnected():
         return
     
     network = self.getPeer().getNetwork()
     neighbors = network.getNeighbors(self.getPeer())
     if len(neighbors) > 0:
         for n in neighbors:
             network.removeConnection(self.getPeer(), n)
             self.getPeer().disconnected()
     else:
         self.getPeer().disconnected()
开发者ID:krohan100,项目名称:pydssim,代码行数:14,代码来源:gnutella_protocol.py


示例16: Msg

class Msg(object):
    """
    TODO: Not documenting this class because it may go away.
    """

    def __init__(self, size):
        self.s_e = Semaphore(1)
        self.s_f = Semaphore(0)
        self.s_buf = Array(ct.c_ubyte, size)

    def send(self, func):
        self.s_e.acquire()
        self.s_buf.acquire()
        send_result = func(self.s_buf._obj)
        self.s_buf.release()
        self.s_f.release()
        return send_result

    def recv(self, func):
        self.s_f.acquire()
        self.s_buf.acquire()
        recv_result = func(self.s_buf._obj)
        self.s_buf.release()
        self.s_e.release()
        return recv_result
开发者ID:rupertsmall,项目名称:neon,代码行数:25,代码来源:image.py


示例17: definyPeerTrading

 def definyPeerTrading(self):
     
     value =0;
     peerAux =""
     semaphore = Semaphore()
     semaphore.acquire()
     for peer,trust in self.__peersTrading.iteritems():
         
         if trust >= value:
             value = trust
             peerAux = peer 
         
         
     semaphore.release()
     return (peerAux,value)
开发者ID:lglmoura,项目名称:PyDDSIM,代码行数:15,代码来源:abstract_trading.py


示例18: ProcessControl

class ProcessControl():
    def __init__(self,forks_number):
        self.forks_number=forks_number
        self.semaphore=Semaphore(self.forks_number)
        
    def execute(self,obj,function_to_execute,data):
        self.semaphore.acquire()
        #print("Launching new process")
        p=Process(target=processCall, args=(self.semaphore,obj,function_to_execute,data))
        p.start()
            
    
    def wait(self):
        for i in range(self.forks_number):
            self.semaphore.acquire()
开发者ID:Bijaye,项目名称:codex-backend,代码行数:15,代码来源:ProcessControl.py


示例19: __init__

    def __init__(self, *args, **kwargs):
        self.url = kwargs.get("url")

        if not self.url:
            raise Exception("No URL to gather")

        self.max_depth = kwargs.get("depth", 1)
        self.workers = kwargs.get("workers", 1)
        self.max_errors = kwargs.get("acceptable_errors", None)

        self.out = kwargs.get("out", "/tmp/")
        if not self.out.endswith("/"):
            self.out += "/"
        self.out += "url_gather/"
        if not os.path.exists(self.out):
            os.makedirs(self.out)

        self.collector_file = kwargs.get("collector_file")
        self.collector_class = kwargs.get("collector_class")
        self._load_collector()
        self._gathered_urls = set()

        # initiate multiprocessing resources
        self._pool = Pool(self.workers)
        self._semaphore = Semaphore(self.workers)
        self._manager = Manager()
        self._url_children = self._manager.dict()
        self._url_errors = self._manager.dict()
        self._url_events = {}
开发者ID:luizcapu,项目名称:crawler_exercises,代码行数:29,代码来源:url_gather.py


示例20: __init__

    def __init__(self, num_processes=1):
        # Set up sync primitives, to communicate with the spawned children
        self.num_processes = num_processes

        # This semaphore is used as a "worker pool guard" to keep the number
        # of spawned workers in the pool to the specified maximum (and block
        # the .spawn_child() call after that)
        self._semaphore = Semaphore(num_processes)

        # This array of integers represents a slot per worker and holds the
        # actual pids (process ids) of the worker's children.  Initially, the
        # array-of-pids is all zeroes.  When a new child is spawned, the pid
        # is written into the slot.  WHen a child finishes, it resets its own
        # slot to 0 again, effectively freeing up the slot (and allowing new
        # children to be spawned).
        self._pids = Array('i', [0] * num_processes)

        # This array of integers also represents a slot per worker and also
        # holds the actual pids of the worker's children.  The difference with
        # _pids, however, is that this array's slots don't get reset
        # immediately when the children end.  In order for Unix subprocesses
        # to actually disappear from the process list (and freeing up the
        # memory), they need to be waitpid()'ed for by the parent process.
        # When each new child is spawned, it waitpid()'s for the (finished)
        # child that was previously in that slot before it claims the new
        # slot.  This mainly avoids ever-growing process lists and slowly
        # growing the memory footprint.
        self._waitfor = Array('i', [0] * num_processes)

        # This array of booleans represent workers that are in their idle
        # state (i.e. they are waiting for work).  During this time, it is
        # safe to terminate them when the user requests so.  Once they start
        # processing work, they flip their idle state and won't be terminated
        # while they're still doing work.
        self._idle = Array('b', [False] * num_processes)
开发者ID:aantonop,项目名称:new_workers,代码行数:35,代码来源:forking.py



注:本文中的multiprocessing.Semaphore类示例由纯净天空整理自Github/MSDocs等源码及文档管理平台,相关代码片段筛选自各路编程大神贡献的开源项目,源码版权归原作者所有,传播和使用请参考对应项目的License;未经允许,请勿转载。


鲜花

握手

雷人

路过

鸡蛋
该文章已有0人参与评论

请发表评论

全部评论

专题导读
上一篇:
Python multiprocessing.SimpleQueue类代码示例发布时间:2022-05-27
下一篇:
Python multiprocessing.Queue类代码示例发布时间:2022-05-27
热门推荐
阅读排行榜

扫描微信二维码

查看手机版网站

随时了解更新最新资讯

139-2527-9053

在线客服(服务时间 9:00~18:00)

在线QQ客服
地址:深圳市南山区西丽大学城创智工业园
电邮:jeky_zhao#qq.com
移动电话:139-2527-9053

Powered by 互联科技 X3.4© 2001-2213 极客世界.|Sitemap