4

Consider two MATLAB m files named profile.m and try_parfor.m as follows.

% profile.m
function profile()

    olddir = pwd();
    cd(tempdir);
    fprintf('\nCurrent directory: %s\n', pwd());

    func = @(x) fun(x, 0);

    addpath(fullfile(fileparts(mfilename('fullpath')), 'try_parfor'));
    try_parfor(func);

    cd(olddir);
    fprintf('\nCurrent directory: %s\n', pwd());

    fprintf('\nSuccess!\n');

end


function y = fun(x, z)
    y = x + z;
end
% try_parfor.m
function try_parfor(func)

    fprintf('\nEntering try_parfor\n');
    fprintf('\nCurrent directory: %s\n', pwd());

    parfor i = 1:2
        fprintf("\n%d\n", feval(func, i));
    end

    fprintf('\nExiting try_parfor\n');

end

If you have MATLAB R2023a--R2025a with the Parallel Computing Toolbox installed, try the following.

  1. Place profile.m in a directory named test.
  2. Place try_parfor.m in the try_parfor subdirectory of test.
  3. Open MATLAB and change the current directory to test.
  4. Run the following command in the MATLAB command window:
profile

It should fail with an error like the following.

 Analyzing and transferring files to the workers ...done.
 {Error using try_parfor (line 6)
 The source code
 (/home/runner/work/test_matlab/test_matlab/test_parfor/try_parfor/try_parfor.m)
 for the parfor-loop that is trying to execute on the worker could not be found.

 Error in profile (line 10)
     try_parfor(func);
     ^^^^^^^^^^^^^^^^^

 Caused by:
     Unrecognized function or variable 'fun'.
     Worker unable to find file.
         Unrecognized function or variable 'fun'.
 }
 exit status 1

Question: Is this failure expected or a bug?

I understand that the failure must depend on the fact that MATLAB has a built-in function named "profile", which is shadowed by "profile.m". Needless to say, this is bad practice, but it is not clear to me why it should lead to such a failure.

As a comparison, rename profile.m to test.m and revise its function name accordingly as follows. Then the MATLAB command

test

will finish successfully.

% test.m
function test()

    olddir = pwd();
    cd(tempdir);
    fprintf('\nCurrent directory: %s\n', pwd());

    func = @(x) fun(x, 0);

    addpath(fullfile(fileparts(mfilename('fullpath')), 'try_parfor'));
    try_parfor(func);

    cd(olddir);
    fprintf('\nCurrent directory: %s\n', pwd());

    fprintf('\nSuccess!\n');

end


function y = fun(x, z)
    y = x + z;
end

See https://github.com/zequipe/test_matlab/tree/master/test_parfor for a reproducer of the issue described above, and https://github.com/zequipe/test_matlab/actions/workflows/test_parfor.yml for a GitHub Actions workflow reproducing it. See https://www.mathworks.com/matlabcentral/answers/2180029-an-issue-with-parfor-in-matlab-r2023a-r2025a for the same question on MATLAB Answers.

Update (made after the very nice and insightful answer & comments by @Edric): It is noteworthy that, as far as I can test, the failure does not occur on MATLAB R2021a—R2022b, as illustrated by my experiment using GitHub actions. However, Edric observes the failure on R2021b. I don’t have a physical machine with MATLAB R2021a—R2022b, and I cannot confirm whether this difference is due to something related to the setup of MATLAB on GitHub Actions. It would be nice if others could conduct a test if machines are available.

1 Answer 1

4

The reasons this turns out not to work are somewhat complicated. Basically, what's happening is that when you send a function_handle from the client to the workers, it needs to be serialized - which is basically equivalent to calling save and then load. During this process, if your function name happens to shadow some other function, then when the serialized form is loaded, the path function is taking precedence. (When there is no shadowing function, the original path to the function is used, despite it not being on the MATLAB path). I don't the doc covers all the ins and outs of serializing function_handle, but it does tell you about the functions function, which can show a little about what's going on.

TLDR: addAttachedFiles(gcp(), [mfilename('fullpath'), '.m']) inside "profile.m" fixes things.

Sign up to request clarification or add additional context in comments.

7 Comments

Thank you @Edric for this very insightful answer. As far as I know (see github.com/zequipe/test_matlab/actions/runs/17820370906 ), this issue does not exist on MATLAB R2021a—R2022b. Is this logical?
I see the same error with R2021b. Analysis of auto-attached files (hopefully) improves as we make changes, but I'm not aware of anything that would address this.
Another question: is there a way to fix it by doing something in try_parfor.m, so that it works regardless of the name of the file calling it? In practice, try_parfor.m corresponds to a package developed by me, and profile.m is a use case by the users. I cannot control the latter. Thanks @Edric
Hm, you could look into functions(func) inside try_parfor, and then use addAttachedFiles using that.
Great! I tried putting addAttachedFiles(gcp(), functions(func).file) (functions(func).file is resolved to profile.m) in try_parfor.m right before the parfor loop, and it works! However, this seems to work only if fun is defined in profile.m (easy to understand) or in an m file under the same directory as profile.m (why??), but it does not work it fun is defined in another directory, e.g., test/private (understandable). Is there any solution in the last case? Thank you @Edric.
"I cannot control the latter." Yes, you can. In the documentation for your package, tell your users not to do this. Or describe this potential error message, and how to fix it.
How would you describe "this" in "not to do this"? If you think about it, "this" is not an issue with any particular package, but a generic problem with any package that uses parfor. It is MathWorks who should document it or fix it.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.