memshare

KO | EN

gitlite — search

#javascript #hacktoberfest #react #llm #go #golang #android #machine-learning #rust #linux #nodejs #java #vue #pytorch

(★ 15)

This project extends R with a mechanism for efficient parallel data access by utilizing C++ shared memory. Large data objects can be accessed and manipulated directly from R without redundant copying, providing both speed and memory efficiency.

memshare 최신버젼 다운로드

최종 버전 다운로드 (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

[![CRAN_Status_Badge](http://www.r-pkg.org/badges/version/memshare)](https://cran.r-project.org/package=memshare)
[![CRAN RStudio mirror downloads](https://cranlogs.r-pkg.org/badges/grand-total/memshare?color=blue)](https://r-pkg.org/pkg/memshare)
[![CRAN RStudio mirror downloads](https://cranlogs.r-pkg.org/badges/last-month/memshare?color=green)](https://r-pkg.org/pkg/memshare)
[![R-CMD-check](https://github.com/Mthrun/memshare/actions/workflows/R-CMD-check.yaml/badge.svg)](https://github.com/Mthrun/memshare/actions/workflows/R-CMD-check.yaml)

# memshare: Shared memory multithreading in R via C++17, with zero-copy access through ALTREP data structures.

---

# Overview

`memshare` enables multicore computation in R without redundant memory copies. Large vectors, matrices, or lists are stored once in shared memory and exposed to R processes as **ALTREP views**. This allows workers in a PSOCK cluster to operate on the same physical data while avoiding serialization overhead.

Key features:

- Shared memory allocation via C++17 back-end (`shm_open` on Unix, `MapViewOfFile` on Windows).
- ALTREP wrappers so R sees shared objects as native vectors or matrices.
- High-level parallels to `parallel::parApply` and `parallel::parLapply`:
  - `memApply()` — apply a function row/column-wise over a matrix in shared memory.
  - `memLapply()` — apply a function element-wise over a list in shared memory.
- Automatic cleanup of shared objects when all views are released or when the package is unloaded.
- Tested on Linux, macOS, and Windows.

---

# Installation

From CRAN:

```r
install.packages("memshare")
```

From GitHub (development version):

```r
remotes::install_github("mthrun/memshare")
```

System requirements: R ≥ 4.0 and a C++17 compiler.

# Quick start

## Example 1: Parallel correlations with a matrix

```r
library(memshare)
library(parallel)

set.seed(1)
n = 10000
p = 2000

X = matrix(rnorm(n * p), n, p)
y = rnorm(n)

res = memApply(
  X = X, MARGIN = 2,
  FUN = function(v, y) cor(v, y),
  VARS = list(y = y)
)
str(res)
```

## Example 2: List operations

```r
library(memshare)
library(parallel)

list_length = 1000
matrix_dim = 100

ListV = lapply(
     1:list_length,
     function(i) matrix(rnorm(matrix_dim * matrix_dim),
     nrow = matrix_dim, ncol = matrix_dim))

y = rnorm(matrix_dim)

namespace = "ID123"
res = memshare::memLapply(ListV, function(el, y) {
   el %*% y
}, NAMESPACE=namespace, VARS=list(y=y), MAX.CORES = 1)

```

Each element el of ListV is multiplied by y in parallel. The list resides once in shared memory.

## Concepts

- Pages: memory regions owned by the current R session that loaded the package.

- Views: ALTREP wrappers exposing shared memory variables (read/write capable).

- Namespaces: string identifiers defining a shared memory context across sessions.

When the package is detached, all handles and associated shared memory pages are released, unless another R process still holds references.

## Shared-memory lifecycle

`memshare` exposes explicit lifecycle functions so you can control when data is
placed in shared memory and when it is freed.

### `registerVariables(namespace, variableList)`
Allocate shared memory and copy R objects (matrices or vectors, or lists for `memLapply`) into it.
- `namespace`: character(1). Identifier of the shared memory context shared across processes.
- `variableList`: a **named** list of objects to register. Names become the keys under which
  you can later retrieve views.

**Example**

```r
library(memshare)

ns <- "my_namespace"
X  <- matrix(rnorm(1e4), 100, 100)
y  <- rnorm(100)

registerVariables(ns, list(X = X, y = y))
# Now X and y live once in shared memory and can be accessed from other R processes
```

### `releaseVariables(namespace, variableNames)`
Delete variables from the shared memory space. Shared regions are only removed when **no active views remain**.
- `namespace`: character(1) used above.
- `variableNames`: character vector of variable names to free.

**Example**

```r
# After all workers have released their views:
releaseVariables(ns, c("X", "y"))
```

### Working with views: `retrieveViews()` and `releaseViews()`
To avoid duplication, workers attach to shared memory by **views**:
- `retrieveViews(namespace, c("X","y"))` returns ALTREP-backed objects that behave like ordinary R matrices/vectors.
- Always call `releaseViews(namespace, ...)` when finished so that the backing memory can be reclaimed.

**Example (worker-side)**

```r
vlist <- retrieveViews(ns, c("X","y"))
# use vlist$X and vlist$y
releaseViews(ns, c("X","y"))
```

> Tip: `memApply()` and `memLapply()` manage views for you automatically, but the low-level API above is useful for custom workflows.

### Manual

The full manual for users or developers is available here:
[Package documentation](https://CRAN.R-project.org/package=memshare/memshare.pdf)

## References <a name="references"/>

[Thrun/Märte, 2026]  Thrun, M. C., & Märte, J.: Memshare: Memory Sharing for Multicore Computation in R with an Application to Feature Selection by Mutual Information using PDE, R Journal, Vol. 17(4), pp. 306 - 322, doi 10.32614/RJ-2025-043, 2026.

[Thrun et al., 2020] Thrun, M.C., Gehlert, T., & Ultsch, A.: Analyzing the Fine Structure of Distributions, PLOS ONE, 15(10), e0238835, 2020.

[Ultsch, 2005] Ultsch, A.: Pareto Density Estimation: A Density Estimation for Knowledge Discovery, Proceedings of the 28th Annual Conference of the German Classification Society, Springer, 2005

memshare

memshare 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

memshare

memshare 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Download README