FilebackedBlockingQueue

KO | EN

gitlite — search

#python #react #ai #typescript #llm #machine-learning #docker #awesome #cli #kubernetes #awesome-list #security #pytorch #mcp

FilebackedBlockingQueue

(★ 8)

File based blocking queue.

FilebackedBlockingQueue Latest Version Download

Download Latest Version (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

FilebackedBlockingQueue
=======================

File based blocking queue is a simple java library, utilizes MappedByteBuffer to store and retrieve data.
The functionality and complexity is comparable to LinkedBlockingQueue.

* Most recent update to the Queue will be in the file cache and hence an immediate read will be faster. 
* Bigger the queue, we will have to do a seek (but sequential IO) to retrieve data in order.
* Removing am element is same as marking the data location for delete and skipping them while reading. (Hence atomic)
* The filesystem/space is split into multiple segments. Each segment is created on demand as the queue grows.
* The segments are recycled (To reduce the number of FD's in use) after the data is retrieved.
* Segments are discard instead of recycling if the max file system size has reached.
* Supports BlockingQueue API's.

* Note: There is (de-)Serialization overhead.
        When JVM exits or the system crashes the files are not in sync, hence we start with a new segments ignoring the old ones before the crash. 
        The above can be solved by doing Periodic sync (not implemented by default).

Example Code:
=============

FileBackedBlockingQueue<String> queue = new FileBackedBlockingQueue.Builder<String>().directory(TEST_DIR).serializer(new StringSerializer()).build();
        for (int i = 0; i < 2000; i++)
            queue.add(TEST_STRING + i);

(OR)

FileBackedBlockingQueue<Runnable> workQueue = new FileBackedBlockingQueue.Builder<Runnable>().directory(dir).serializer(MessageTask.serializer).build();
        executor = new ThreadPoolExecutor(threadCount, threadCount, 60L, TimeUnit.SECONDS, workQueue);
        if (pattern.matcher(msgContainer.routingKey).matches())
            executor.execute(new MessageTask(msgContainer));

FilebackedBlockingQueue

FilebackedBlockingQueue Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

FilebackedBlockingQueue

FilebackedBlockingQueue Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

Download README