CPython2.7 subprocess.Popen()不忍直视的Unicode中文路径Bug 解决办法

2018-1-28 06:57| 发布者: digitser| 查看: 3996| 评论: 0|原作者: Python

摘要: CPython2.7 subprocess.Popen()不忍直视的Unicode中文路径Bug 解决办法 On Windows, Python2.7 subprocess.Popen() requires the executable name and working directory to be ascii.This is because it calls Crea ...

自动立式分页纸箱赋码系统 ── 全自动专业立式分页瓦楞纸水性油墨贴标喷码检测系统

CPython2.7 subprocess.Popen()不忍直视的Unicode中文路径Bug 解决办法

On Windows, Python2.7 subprocess.Popen() requires the executable name and working directory to be ascii. This is because it calls CreateProcess instead of CreateProcessW.

Python2.7 subprocess.Popen() parameters are Unicode strings. It may support bytes string, but passing bytes to OS functions on Windows is deprecated, especially for filenames.

In Python2, windows implementation of subprocess.Popen(..) and sys.argv use the non unicode ready windows systems call CreateProcess(..)，and does not use GetCommandLineW(..) for sys.argv.

In Python3, windows implementation of subprocess.Popen(..) make use of the correct windows systems calls CreateProcessW(..) starting from 3.0 (see code in 3.0), and sys.argv uses GetCommandLineW(..) starting from 3.3 (see code in 3.3).

百度网盘

https://pan.baidu.com/s/1cKxajG

软件仓库

https://github.com/digitser

https://digitser.sourceforge.io/

https://pan.baidu.com/s/1dGxcM7R

快速高效智能编辑重构批处理 "数字化 Python IDE" 集成开发环境

http://dts.digitser.cn/zh-CN/ide/idepy/index.html

http://forum.digitser.cn/thread-2177-1-1.html

不修正 Python2.7 subprocess.Popen() Unicode 路径原因

It's tricky to fix this issue in Python2.7 because you have to choose which function is used: CreateProcessA() (bytes) or CreateProcessW() (Unicode). To use CreateProcessW(), you have to decode bytes parameter.

Python3.x has os.fsencode()/os.fsdecode() functions, similar functions in C. The "mbcs" Python codec is strict by default, but it now supports any Python error handler. This change changed was improved in each Python 3 release.

Python 2 has PyUnicode_DecodeMBCSStateful() and PyUnicode_EncodeMBCS() which use the default Windows behaviour. using PyUnicode_DecodeMBCSStateful() (or directly MultiByteToWideChar) + CreateProcessW() is exactly the same than calling CreateProcessA().

Should support CreateProcessA() and CreateProcessW(), and use one or the other depending on the type of the pararameters?

Such change requires too much work and it is not enough to have a full Unicode support for filenames. Have to fix much more code. Already did all this work in Python3.x (in 3.1, 3.2 and then 3.3). Suggest to upgrade to port the application to Python3.x, if want a full Unicode support. Using Unicode in Python3 is natural and just works fine.

CreateProcessW

以下是 Python3.4 subprocess.Popen 类源代码 _winapi.CreateProcess 部分，要调用的 _winapi.c 代码片段。

开始采用 CreateProcessW，说明 Unicode 中文路径 Bug 问题已彻底修复。

static PyObject *
winapi_CreateProcess(PyObject* self, PyObject* args)
{
BOOL result;
PROCESS_INFORMATION pi;
STARTUPINFOW si;
PyObject* environment;
wchar_t *wenvironment;
wchar_t* application_name;
wchar_t* command_line;
PyObject* process_attributes; /* ignored */
PyObject* thread_attributes; /* ignored */
BOOL inherit_handles;
DWORD creation_flags;
PyObject* env_mapping;
wchar_t* current_directory;
PyObject* startup_info;
if (! PyArg_ParseTuple(args, "ZZOO" F_BOOL F_DWORD "OZO:CreateProcess",
&application_name,
&command_line,
&process_attributes,
&thread_attributes,
&inherit_handles,
&creation_flags,
&env_mapping,
¤t_directory,
&startup_info))
return NULL;
ZeroMemory(&si, sizeof(si));
si.cb = sizeof(si);
/* note: we only support a small subset of all SI attributes */
si.dwFlags = getulong(startup_info, "dwFlags");
si.wShowWindow = (WORD)getulong(startup_info, "wShowWindow");
si.hStdInput = gethandle(startup_info, "hStdInput");
si.hStdOutput = gethandle(startup_info, "hStdOutput");
si.hStdError = gethandle(startup_info, "hStdError");
if (PyErr_Occurred())
return NULL;
if (env_mapping != Py_None) {
environment = getenvironment(env_mapping);
+
−if (! environment)
return NULL;
wenvironment = PyUnicode_AsUnicode(environment);
if (wenvironment == NULL)
{
Py_XDECREF(environment);
return NULL;
}
}
else {
environment = NULL;
wenvironment = NULL;
}
Py_BEGIN_ALLOW_THREADS
result = CreateProcessW(application_name,
command_line,
NULL,
NULL,
inherit_handles,
creation_flags | CREATE_UNICODE_ENVIRONMENT,
wenvironment,
current_directory,
&si,
&pi);
Py_END_ALLOW_THREADS
Py_XDECREF(environment);
if (! result)
return PyErr_SetFromWindowsErr(GetLastError());
return Py_BuildValue("NNkk",
HANDLE_TO_PYNUM(pi.hProcess),
HANDLE_TO_PYNUM(pi.hThread),
pi.dwProcessId,
pi.dwThreadId);
}

复制代码

解决办法

测试了 windll.kernel32.CreateProcessW() 方法，直接 windll.kernel32.CreateProcessW(u"C:\\WINDOWS\\system32\\calc.exe", None, None, None, None, creation_flags, None, None, byref(startupinfo), byref(process_info)) 没问题。

由于水平问题，若要 windll.kernel32.CreateProcessW(u"E:\\Python27\\python.exe **process_file.py", None, None, None, None, creation_flags, None, None, byref(startupinfo), byref(process_info)) 则还是不行。估计得修改 CPython2.7 源代码，个人时间有限，暂时不弄啦。

有人参照 CPython3.x 为 CPython2.7 写了 2 个补丁，分别是 subprocess_fix 和 commandline_fix，但其 Git 地址已失效，且新进程还得考虑使用以下代码 "传递参数"。

http://vaab.blog.kal.fr/2017/03/16/fixing-windows-python-2-7-unicode-issue-with-subprocesss-popen/

找了好久，那 2 个两个补丁也没找到，还是用 Python3.x 吧。