Commit fresh-releases/main - python-irodsclient

+41

-4

CHANGELOG.rst less more

0	0	Changelog
1	1	=========
2	2
	3	v1.1.8 (2023-05-18)
	4	-------------------
	5	- [#450] test for setting/getting comments [Daniel Moore]
	6	- [#377] fix iRODSSession connection timeout [Daniel Moore]
	7	- [#450] Get property comments from replica object [Gwenael Leysour de Rohello]
	8	- [#448] protect against bad parameters in modify_password [Daniel Moore]
	9	- [#442] allow non-default string types in AVU fields [Daniel Moore]
	10	- [#443] add NotLike (GenQuery 'NOT LIKE') operator [Daniel Moore]
	11
	12	v1.1.7 (2023-03-28)
	13	-------------------
	14	- [#435] unregister can target a single replica [Daniel Moore]
	15	- [#434] metadata calls now require AVU fields to be nonzero-length strings [Daniel Moore]
	16	- [#431][irods/irods#6921] filter user_id results from R_OBJT_ACCESS through IDs still in R_USER_MAIN [Daniel Moore]
	17	- [#3] acls.set needs admin=True for some tests [Daniel Moore]
	18	- [#3] compatibility bump to iRODS 4.3.1 [Daniel Moore]
	19	- [#426][#428][#429] groupadmin capabilities update [Daniel Moore]
	20
	21	v1.1.6 (2023-01-18)
	22	-------------------
	23	- [#420][#422] present appropriate iRODSAccess.codes, in sorted order [Daniel Moore]
	24	- [#420] store integer codes & strings for access levels [Daniel Moore]
	25	- [#418] raise error in test for IRODS_VERSION mismatch [Daniel Moore]
	26	- [#379] define RE_RUNTIME_ERROR exception [Daniel Moore]
	27	- [#400] more advanced iRODSException representation [Daniel Moore]
	28	- [#392] add iRODSResource properties: parent_name, parent_id, hierarchy_string [Daniel Moore]
	29	- [#243] enable RESC_HIER_STR_KW and RESC_NAME_KW in data open() [Daniel Moore]
	30	- [#395][#396] test of acls manager [Daniel Moore]
	31	- [#396] Introduce "acls" manager and deprecate "permissions" [Daniel Moore]
	32	- [#395] include user_type in permissions [Daniel Moore]
	33	- [#410] ensure a call to iRODSSession.cleanup() at interpreter exit [Daniel Moore]
	34	- [#406] correctly generate ssl context [Daniel Moore]
	35	- [#404] Fix password_obfuscation in Windows [J.P. Mc Farland]
	36	- [#374] Use alternate endpoint for groupadmin requests [Martin Jaime Flores Jr]
	37	- [#5] minor README fix: XML_Parser_Type code sample [Sietse Snel]
	38	- [#5] adds module loading to RErrorStack example [John Constable]
	39
3	40	v1.1.5 (2022-09-21)
4	41	-------------------
5		- [#383] correct logical path normalization
6		- [#369] remove dynamic generation of message classes
7		- [#386][#389] only load timestamps when requested
8		- [#386] initial change to add create and modify times for metadata
	42	- [#383] correct logical path normalization [Daniel Moore]
	43	- [#369] remove dynamic generation of message classes [Daniel Moore]
	44	- [#386][#389] only load timestamps when requested [Daniel Moore]
	45	- [#386] initial change to add create and modify times for metadata [Paul Borgermans]
9	46
10	47	v1.1.4 (2022-06-29)
11	48	-------------------

+116

-50

README.rst less more

75	75	Establishing a (secure) connection
76	76	----------------------------------
77	77
	78	One way of starting a session is to pass iRODS credentials as keyword arguments:
	79
	80	>>> from irods.session import iRODSSession
	81	>>> with iRODSSession(host='localhost', port=1247, user='bob', password='1234', zone='tempZone') as session:
	82	... # workload
	83	...
	84	>>>
	85
	86	If you're an administrator acting on behalf of another user:
	87
	88	>>> from irods.session import iRODSSession
	89	>>> with iRODSSession(host='localhost', port=1247, user='rods', password='1234', zone='tempZone',
	90	client_user='bob', client_zone='possibly_another_zone') as session:
	91	... # workload
	92	...
	93	>>>
	94
	95	If no ``client_zone`` is provided, the ``zone`` parameter is used in its place.
	96
78	97	Using environment files (including any SSL settings) in ``~/.irods/``:
79	98
80	99	>>> import os

85	104	... except KeyError:
86	105	... env_file = os.path.expanduser('~/.irods/irods_environment.json')
87	106	...
88		>>> ssl_context = ssl.create_default_context(purpose=ssl.Purpose.SERVER_AUTH, cafile=None, capath=None, cadata=None)
89		>>> ssl_settings = {'ssl_context': ssl_context}
	107	>>> ssl_settings = {} # Or, optionally: {'ssl_context': <user_customized_SSLContext>}
90	108	>>> with iRODSSession(irods_env_file=env_file, **ssl_settings) as session:
91	109	... # workload
92	110	...
93	111	>>>
94	112
95		Passing iRODS credentials as keyword arguments:
96
	113	In the above example, an SSL connection can be made even if no 'ssl_context' argument is specified,
	114	in which case the Python client internally generates its own SSLContext object to best
	115	match the iRODS SSL configuration parameters (such as "irods_ssl_ca_certificate_file",
	116	etc.) used to initialize the iRODSSession. Those parameters can be given either
	117	in the environment file, or in the iRODSSession constructor call as shown by the next example.
	118
	119	A pure Python SSL session (without a local `env_file`) requires a few more things defined:
	120
	121	>>> import ssl
97	122	>>> from irods.session import iRODSSession
98		>>> with iRODSSession(host='localhost', port=1247, user='bob', password='1234', zone='tempZone') as session:
99		... # workload
100		...
101		>>>
102
103		If you're an administrator acting on behalf of another user:
104
105		>>> from irods.session import iRODSSession
106		>>> with iRODSSession(host='localhost', port=1247, user='rods', password='1234', zone='tempZone',
107		client_user='bob', client_zone='possibly_another_zone') as session:
108		... # workload
109		...
110		>>>
111
112		If no ``client_zone`` is provided, the ``zone`` parameter is used in its place.
113
114		A pure Python SSL session (without a local `env_file`) requires a few more things defined:
115
116		>>> import ssl
117		>>> from irods.session import iRODSSession
118		>>> ssl_context = ssl.create_default_context(purpose=ssl.Purpose.SERVER_AUTH, cafile='CERTNAME.crt', capath=None, cadata=None)
119	123	>>> ssl_settings = {'client_server_negotiation': 'request_server_negotiation',
120	124	... 'client_server_policy': 'CS_NEG_REQUIRE',
121	125	... 'encryption_algorithm': 'AES-256-CBC',
122	126	... 'encryption_key_size': 32,
123	127	... 'encryption_num_hash_rounds': 16,
124		... 'encryption_salt_size': 8,
125		... 'ssl_context': ssl_context}
126		>>>
127		>>> with iRODSSession(host='HOSTNAME_DEFINED_IN_CAFILE_ABOVE', port=1247, user='bob', password='1234', zone='tempZone', **ssl_settings) as session:
	128	... 'encryption_salt_size': 8,
	129	... 'ssl_context': ssl_context
	130	... 'ssl_verify_server': 'cert',
	131	... 'ssl_ca_certificate_file': '/etc/irods/ssl/irods.crt'
	132	... }
	133
	134	If necessary, a user may provide a custom SSLContext object; although, as of release v1.1.6, this will rarely be required:
	135
	136	>>> ssl_settings ['ssl_context'] = ssl.create_default_context(purpose=ssl.Purpose.SERVER_AUTH, # ... other options
	137	... )
	138
	139	At this point, we are ready to instantiate and use the session:
	140
	141	>>> with iRODSSession(host='irods-provider', port=1247, user='bob', password='1234', zone='tempZone', **ssl_settings) as session:
128	142	... # workload
129		>>>
130
	143
	144	Note that the :code:`irods_` prefix is unnecessary when providing the :code:`encryption_` and :code:`ssl_` options directly to the
	145	constructor as keyword arguments, even though it is required when they are placed in the environment file.
131	146
132	147	Maintaining a connection
133	148	------------------------

308	323	Additionally, if a freshly created irods.message.RErrorStack instance is given, information can be returned and read by
309	324	the client:
310	325
	326	>>> from irods.message import RErrorStack
311	327	>>> r_err_stk = RErrorStack()
312	328	>>> warn = None
313	329	>>> try: # Here, data_obj has one replica, not yet checksummed.

509	525	::
510	526
511	527	from irods.message import (XML_Parser_Type, ET)
512		ET( XML_Parser.QUASI_XML, session.server_version )
	528	ET( XML_Parser_Type.QUASI_XML, session.server_version )
513	529
514	530	Two dedicated environment variables may also be used to customize the Python client's XML parsing behavior via the
515	531	setting of global defaults during start-up.

1047	1063	iRODS tracks groups and users using two tables, R_USER_MAIN and R_USER_GROUP.
1048	1064	Under this database schema, all "user groups" are also users:
1049	1065
1050		>>> from irods.models import User, UserGroup
	1066	>>> from irods.models import User, Group
1051	1067	>>> from pprint import pprint
1052		>>> pprint(list( [ (x[User.id], x[User.name]) for x in session.query(User) ] ))
	1068	>>> pprint(list((x[User.id], x[User.name]) for x in session.query(User)))
1053	1069	[(10048, 'alice'),
1054	1070	(10001, 'rodsadmin'),
1055	1071	(13187, 'bobby'),

1066	1082	>>> [x[User.name] for x in groups]
1067	1083	['collab', 'public', 'rodsadmin', 'empty']
1068	1084
1069		Since we can instantiate iRODSUserGroup and iRODSUser objects directly from the rows of
	1085	Since we can instantiate iRODSGroup and iRODSUser objects directly from the rows of
1070	1086	a general query on the corresponding tables, it is also straightforward to trace out
1071	1087	the groups' memberships:
1072	1088
1073		>>> from irods.user import iRODSUser, iRODSUserGroup
1074		>>> grp_usr_mapping = [ (iRODSUserGroup ( session.user_groups, result), iRODSUser (session.users, result)) \
1075		... for result in session.query(UserGroup,User) ]
	1089	>>> from irods.user import iRODSUser, iRODSGroup
	1090	>>> grp_usr_mapping = [ (iRODSGroup(session.groups, result), iRODSUser(session.users, result)) \
	1091	... for result in session.query(Group,User) ]
1076	1092	>>> pprint( [ (x,y) for x,y in grp_usr_mapping if x.id != y.id ] )
1077		[(<iRODSUserGroup 10045 collab>, <iRODSUser 10048 alice rodsuser tempZone>),
1078		(<iRODSUserGroup 10001 rodsadmin>, <iRODSUser 10003 rods rodsadmin tempZone>),
1079		(<iRODSUserGroup 10002 public>, <iRODSUser 10003 rods rodsadmin tempZone>),
1080		(<iRODSUserGroup 10002 public>, <iRODSUser 10048 alice rodsuser tempZone>),
1081		(<iRODSUserGroup 10045 collab>, <iRODSUser 13187 bobby rodsuser tempZone>),
1082		(<iRODSUserGroup 10002 public>, <iRODSUser 13187 bobby rodsuser tempZone>)]
	1093	[(<iRODSGroup 10045 collab>, <iRODSUser 10048 alice rodsuser tempZone>),
	1094	(<iRODSGroup 10001 rodsadmin>, <iRODSUser 10003 rods rodsadmin tempZone>),
	1095	(<iRODSGroup 10002 public>, <iRODSUser 10003 rods rodsadmin tempZone>),
	1096	(<iRODSGroup 10002 public>, <iRODSUser 10048 alice rodsuser tempZone>),
	1097	(<iRODSGroup 10045 collab>, <iRODSUser 13187 bobby rodsuser tempZone>),
	1098	(<iRODSGroup 10002 public>, <iRODSUser 13187 bobby rodsuser tempZone>)]
1083	1099
1084	1100	(Note that in general queries, fields cannot be compared to each other, only to literal constants; thus
1085	1101	the '!=' comparison in the Python list comprehension.)

1091	1107	members, so it doesn't show up in our final list.
1092	1108
1093	1109
	1110	Group Administrator Capabilities
	1111	--------------------------------
	1112	With v1.1.7, PRC acquires the full range of abilities possessed by the igroupadmin command.
	1113
	1114	Firstly, a groupadmin may invoke methods to create groups, and may add users to, or remove them from, any
	1115	group to which they themselves belong:
	1116
	1117	>>> session.groups.create('lab')
	1118	>>> session.groups.addmember('lab',session.username) # allow self to administer group
	1119	>>> session.groups.addmember('lab','otheruser')
	1120	>>> session.groups.removemember('lab','otheruser')
	1121
	1122	In addition, a groupadmin may also create accounts for new users and enable their logins by initializing
	1123	a native password for them:
	1124
	1125	>>> session.users.create_with_password('alice', 'change_me')
	1126
	1127
	1128	iRODS Permissions (ACLs)
	1129	------------------------
	1130
	1131	The :code:`iRODSAccess` class offers a convenient dictionary interface mapping iRODS permission
	1132	strings to the corresponding integer codes:
	1133
	1134	>>> from irods.access import iRODSAccess
	1135	>>> iRODSAccess.keys()
	1136	['null', 'read_metadata', 'read_object', 'create_metadata', 'modify_metadata', 'delete_metadata', 'create_object', 'modify_object', 'delete_object', 'own']
	1137	>>> WRITE = iRODSAccess.to_int('modify_object')
	1138
	1139	Armed with that, we can then query on all data objects with ACL's that allow our user to write them:
	1140
	1141	>>> from irods.models import (DataObject, User, DataAccess)
	1142	>>> data_objects_writable = list(session.query(DataObject, User, DataAccess).filter(User.name == session.username, DataAccess.type >= WRITE))
	1143
	1144	Finally, we can also access the list of permissions available through a given session object via the :code:`available_permissions` property.
	1145	Note that -- in keeping with changes in iRODS server 4.3 -- the permissions list will be longer, as appropriate, for session objects connected to
	1146	the more recent servers; and also that the embedded spaces in some 4.2 permission strings will be replaced by underscores in 4.3 and later.
	1147
	1148	>>> session.server_version
	1149	(4, 2, 11)
	1150	>>> session.available_permissions.items()
	1151	[('null', 1000), ('read object', 1050), ('modify object', 1120), ('own', 1200)]
	1152
	1153
1094	1154	Getting and setting permissions
1095	1155	-------------------------------
1096	1156

1107	1167
1108	1168	>>> from irods.access import iRODSAccess
1109	1169	>>> for c in q:
1110		... session.permissions.set( iRODSAccess('read', c[Collection.name], 'alice', # 'otherZone' # zone optional
	1170	... session.acls.set( iRODSAccess('read', c[Collection.name], 'alice', # 'otherZone' # zone optional
1111	1171	... ))
1112	1172
1113		We can also query on access type using its numeric value, which will seem more natural to some:
1114
1115		>>> OWN = 1200; MODIFY = 1120 ; READ = 1050
1116		>>> from irods.models import DataAccess, DataObject, User
1117		>>> data_objects_writable = list(session.query(DataObject,DataAccess,User)).filter(User.name=='alice', DataAccess.type >= MODIFY)
	1173	A call to :code:`session.acls.get(c)` -- with :code:`c` being the result of :code:`sessions.collections.get(c[Collection.name])` --
	1174	would then verify the desired change had taken place (as well as list all ACLs stored in the catalog for that collection).
	1175
	1176	One last note on permissions: The older access manager, :code:`<session>.permissions`, produced inconsistent results when the :code:`get()`
	1177	method was invoked with the parameter :code:`report_raw_acls` set (or defaulting) to :code:`False`. Specifically, collections would exhibit the
	1178	"non-raw-ACL" behavior of reporting individual member users' permissions as a by-product of group ACLs, whereas data objects would not.
	1179
	1180	In release v1.1.6, we move to correct this inconsistency by introducing the synonym :code:`<session>.acls` that acts almost identically
	1181	like :code:`<session>.permissions`, except that the :code:`<session>.acls.get(...)` method does not accept the :code:`report_raw_acls` parameter. When we need to detect users' permissions independent of their access to an object via group membership, this can be achieved with another query.
	1182
	1183	:code:`<session>.permissions` is therefore deprecated and, in v2.0.0, will be removed in favor of :code:`<session>.acls`.
1118	1184
1119	1185
1120	1186	Managing users

+3

-2

debian/changelog less more

0		python-irodsclient (1.1.5-1) UNRELEASED; urgency=medium
	0	python-irodsclient (1.1.8-1) UNRELEASED; urgency=medium
1	1
2	2	* Update standards version to 4.6.2, no changes needed.
3	3	* New upstream release.
	4	* New upstream release.
4	5
5		-- Debian Janitor <janitor@jelmer.uk> Wed, 11 Jan 2023 04:24:14 -0000
	6	-- Debian Janitor <janitor@jelmer.uk> Mon, 19 Jun 2023 04:09:46 -0000
6	7
7	8	python-irodsclient (0.8.1-3) unstable; urgency=medium
8	9

+86

-3

irods/access.py less more

0		class iRODSAccess(object):
	0	import collections
	1	import copy
	2	import six
1	3
2		def __init__(self, access_name, path, user_name='', user_zone=''):
	4	class _Access_LookupMeta(type):
	5	def __getitem__(self, key): return self.codes[key]
	6	def keys(self): return list(self.codes.keys())
	7	def values(self): return list(self.codes[k] for k in self.codes.keys())
	8	def items(self): return list(zip(self.keys(),self.values()))
	9
	10	class iRODSAccess(six.with_metaclass(_Access_LookupMeta)):
	11
	12	@classmethod
	13	def to_int(cls,key):
	14	return cls.codes[key]
	15
	16	@classmethod
	17	def to_string(cls,key):
	18	return cls.strings[key]
	19
	20	codes = collections.OrderedDict((key_,value_) for key_,value_ in sorted(dict(
	21	# copied from iRODS source code in
	22	# ./server/core/include/irods/catalog_utilities.hpp:
	23	null = 1000,
	24	execute = 1010,
	25	read_annotation = 1020,
	26	read_system_metadata = 1030,
	27	read_metadata = 1040,
	28	read_object = 1050,
	29	write_annotation = 1060,
	30	create_metadata = 1070,
	31	modify_metadata = 1080,
	32	delete_metadata = 1090,
	33	administer_object = 1100,
	34	create_object = 1110,
	35	modify_object = 1120,
	36	delete_object = 1130,
	37	create_token = 1140,
	38	delete_token = 1150,
	39	curate = 1160,
	40	own = 1200
	41	).items(),key=lambda _:_[1]) if key_ in (
	42	# These are copied from ichmod help text.
	43	'own',
	44	'delete_object',
	45	'write', 'modify_object',
	46	'create_object',
	47	'delete_metadata',
	48	'modify_metadata',
	49	'create_metadata',
	50	'read', 'read_object',
	51	'read_metadata',
	52	'null'
	53	)
	54	)
	55
	56	strings = collections.OrderedDict((number,string) for string,number in codes.items())
	57
	58	def __init__(self, access_name, path, user_name='', user_zone='', user_type=None):
3	59	self.access_name = access_name
4	60	self.path = path
5	61	self.user_name = user_name
6	62	self.user_zone = user_zone
	63	self.user_type = user_type
	64
	65	def copy(self, decanonicalize = False):
	66	other = copy.deepcopy(self)
	67	if decanonicalize:
	68	replacement_string = { 'read object':'read',
	69	'read_object':'read',
	70	'modify object':'write',
	71	'modify_object':'write'}.get(self.access_name)
	72	other.access_name = replacement_string if replacement_string is not None \
	73	else self.access_name
	74	return other
7	75
8	76	def __repr__(self):
9		return "<iRODSAccess {access_name} {path} {user_name} {user_zone}>".format(**vars(self))
	77	object_dict = vars(self)
	78	access_name = self.access_name.replace(' ','_')
	79	user_type_hint = ("({user_type})" if object_dict.get('user_type') is not None else "").format(**object_dict)
	80	return "<iRODSAccess {0} {path} {user_name}{1} {user_zone}>".format(access_name,
	81	user_type_hint,
	82	**object_dict)
	83
	84	class _iRODSAccess_pre_4_3_0(iRODSAccess):
	85	codes = collections.OrderedDict(
	86	(key.replace('_',' '),value) for key,value in iRODSAccess.codes.items() if key in (
	87	'own',
	88	'write', 'modify_object',
	89	'read', 'read_object',
	90	'null'
	91	))
	92	strings = collections.OrderedDict((number,string) for string,number in codes.items())

+6

-0

irods/column.py less more

57	57
58	58	def __init__(self, query_key, value):
59	59	super(Like, self).__init__('like', query_key, value)
	60
	61
	62	class NotLike(Criterion):
	63
	64	def __init__(self, query_key, value):
	65	super(NotLike, self).__init__('not like', query_key, value)
60	66
61	67
62	68	class Between(Criterion):

+17

-10

irods/connection.py less more

21	21	PluginAuthMessage, ClientServerNegotiation, Error, GetTempPasswordOut)
22	22	from irods.exception import (get_exception_by_code, NetworkException, nominal_code)
23	23	from irods.message import (PamAuthRequest, PamAuthRequestOut)
	24
24	25
25	26
26	27	ALLOW_PAM_LONG_TOKENS = True # True to fix [#279]

159	160	return False
160	161	return False
161	162
	163	@staticmethod
	164	def make_ssl_context(irods_account):
	165	check_hostname = getattr(irods_account,'ssl_verify_server','hostname')
	166	CAfile = getattr(irods_account,'ssl_ca_certificate_file',None)
	167	CApath = getattr(irods_account,'ssl_ca_certificate_path',None)
	168	verify = ssl.CERT_NONE if (None is CAfile is CApath) else ssl.CERT_REQUIRED
	169	# See https://stackoverflow.com/questions/30461969/disable-default-certificate-verification-in-python-2-7-9/49040695#49040695
	170	ctx = ssl.create_default_context(ssl.Purpose.SERVER_AUTH, cafile=CAfile, capath=CApath)
	171	# Note: check_hostname must be assigned prior to verify_mode property or Python library complains!
	172	ctx.check_hostname = (check_hostname.startswith('host') and verify != ssl.CERT_NONE)
	173	ctx.verify_mode = verify
	174	return ctx
	175
162	176	def ssl_startup(self):
163	177	# Get encryption settings from client environment
164	178	host = self.account.host

167	181	hash_rounds = self.account.encryption_num_hash_rounds
168	182	salt_size = self.account.encryption_salt_size
169	183
170		# Get or create SSL context
171	184	try:
172	185	context = self.account.ssl_context
173	186	except AttributeError:
174		CA_file = getattr(self.account, 'ssl_ca_certificate_file', None)
175		verify_server_mode = getattr(self.account,'ssl_verify_server', 'hostname')
176		if verify_server_mode == 'none':
177		CA_file = None
178		context = ssl.create_default_context(ssl.Purpose.SERVER_AUTH, cafile=CA_file)
179		if CA_file is None:
180		context.check_hostname = False
181		context.verify_mode = 0 # VERIFY_NONE
	187	self.account.ssl_context = context = self.make_ssl_context(self.account)
182	188
183	189	# Wrap socket with context
184		wrapped_socket = context.wrap_socket(self.socket, server_hostname=host)
	190	wrapped_socket = context.wrap_socket(self.socket,
	191	server_hostname=(host if context.check_hostname else None))
185	192
186	193	# Initial SSL handshake
187	194	wrapped_socket.do_handshake()

+2

-1

irods/data_object.py less more

68	68	r[DataObject.path],
69	69	r[DataObject.resc_hier],
70	70	checksum=r[DataObject.checksum],
71		size=r[DataObject.size]
	71	size=r[DataObject.size],
	72	comments=r[DataObject.comments]
72	73	) for r in replicas]
73	74	self._meta = None
74	75

+77

-6

irods/exception.py less more

2	2
3	3
4	4	from __future__ import absolute_import
	5	from __future__ import print_function
	6	import errno
	7	import numbers
	8	import os
5	9	import six
6		import numbers
	10	import sys
7	11
8	12
9	13	class PycommandsException(Exception):

34	38	pass
35	39
36	40
37		class UserGroupDoesNotExist(DoesNotExist):
38		pass
39
	41	class GroupDoesNotExist(DoesNotExist):
	42	pass
	43
	44	# NOTE: Everything of the form UserGroup is deprecated.
	45	UserGroupDoesNotExist = GroupDoesNotExist
40	46
41	47	class ResourceDoesNotExist(DoesNotExist):
42	48	pass

60	66
61	67	class iRODSExceptionMeta(type):
62	68	codes = {}
63
	69	positive_code_error_message = "For {name}, a positive code of {attrs[code]} was declared."
64	70	def __init__(self, name, bases, attrs):
65	71	if 'code' in attrs:
	72	if attrs['code'] > 0:
	73	print(self.positive_code_error_message.format(**locals()), file = sys.stderr)
	74	exit(1)
66	75	iRODSExceptionMeta.codes[attrs['code']] = self
67	76
68	77
	78	class Errno:
	79	"""Encapsulates an integer error code from the operating system
	80	and provides a text representation.
	81	"""
	82	def __init__(self,arg0,*_):
	83	"""Initializes an object with an integer error code arg0.
	84	Further arguments are optional and ignored.
	85	"""
	86	self.int_code = arg0
	87
	88	def __repr__(self):
	89	e = self.int_code
	90	try:
	91	return self.__class__.__name__ + repr(tuple([e, errno.errorcode[e], os.strerror(e)]))
	92	except:
	93	# The errno code is unrecognized, so fall through to default representation.
	94	pass
	95	return self.__class__.__name__ + repr(tuple([e,]))
	96
	97	def __int__(self):
	98	return self.int_code
	99
	100
69	101	class iRODSException(six.with_metaclass(iRODSExceptionMeta, Exception)):
70		pass
	102	"""An exception that originates from a server error.
	103	Exception classes that are derived from this base and represent a concrete error, should
	104	store a unique error code (X*1000) in their 'code' attribute, where X < 0.
	105	"""
	106
	107	def __init__(self,*arg):
	108	explicit_errno = None
	109	argl = list(arg)
	110
	111	# For consistency with the text representation, allow initialization with an Errno object
	112	# at the end of the argument list.
	113	# Example: err = UNIX_FILE_OPEN_ERR('message', Errno(13))
	114	# err_copy = eval(repr(err))
	115	if hasattr(self.__class__,'code'):
	116	if argl and isinstance (argl[-1],Errno):
	117	explicit_errno = argl.pop()
	118
	119	super(iRODSException,self).__init__(*argl)
	120
	121	# To properly represent the Errno instance, the "code" attribute should be (-errno) plus the thousands
	122	# attribute stored in the class's code attribute. Because Python honors the instance "code" attribute
	123	# over the class "code" attribute, the following does _not_ modify the class:
	124	if explicit_errno:
	125	self.code -= int(explicit_errno)
	126
	127	def __repr__(self):
	128	args = tuple(self.args)
	129	code = getattr(self,'code',None)
	130	if code is not None:
	131	os_err = abs(code) % 1000
	132	if os_err: args += (Errno(os_err),)
	133	return self.__class__.__name__ + repr(args)
	134
71	135
72	136
73	137	def nominal_code( the_code, THRESHOLD = 1000 ):

1586	1650	code = -1016000
1587	1651
1588	1652
	1653	class RE_RUNTIME_ERROR(RuleEngineException):
	1654	code = -1205000
	1655
	1656
1589	1657	class RE_TYPE_ERROR(RuleEngineException):
1590	1658	code = -1230000
1591	1659

1977	2045	class PHP_OPEN_SCRIPT_FILE_ERR(PHPException):
1978	2046	code = -1602000
1979	2047
	2048	class DIRECT_CHILD_ACCESS(iRODSException):
	2049	code = -1816000
	2050
1980	2051	class PAMException(iRODSException):
1981	2052	pass
1982	2053

+27

-7

irods/manager/access_manager.py less more

13	13
14	14	import six
15	15	import logging
	16	import warnings
16	17
17	18	logger = logging.getLogger(__name__)
18	19

31	32	class AccessManager(Manager):
32	33
33	34	def get(self, target, report_raw_acls = False, **kw):
	35
	36	if not kw.pop('suppress_deprecation_warning',False):
	37	warnings.warn('Use of session_obj.permissions is deprecated in v1.1.6',
	38	DeprecationWarning, stacklevel = 2)
34	39
35	40	if report_raw_acls:
36	41	return self.__get_raw(target, **kw) # prefer a behavior consistent with 'ils -A`

55	60	results = self.sess.query(user_type.name, user_type.zone, access_type.name)\
56	61	.filter(*conditions).all()
57	62
	63	def get_usertype(row):
	64	return self.sess.users.get(row[user_type.name], row[user_type.zone]).type
	65
58	66	return [iRODSAccess(
59	67	access_name=row[access_type.name],
60	68	user_name=row[user_type.name],
	69	user_type=get_usertype(row),
61	70	path=target.path,
62	71	user_zone=row[user_type.zone]
63	72	) for row in results]

75	84	### sample usage: ###
76	85	#
77	86	# user_id_list = [] # simply to store the user id's from the discovered ACL's
78		# session.permissions.get( data_or_coll_target, report_raw_acls = True,
79		# acl_users = user_id_list,
80		# acl_users_transform = lambda u: u.id)
	87	# session.acls.get( data_or_coll_target, acl_users = user_id_list,
	88	# acl_users_transform = lambda u: u.id)
81	89	#
82	90	# -> returns list of iRODSAccess objects mapping one-to-one with ACL's stored in the catalog
83	91

95	103	else:
96	104	raise TypeError
97	105
98		rows = [ r for r in query_func(target.path) ]
	106	# TODO: remove the filtering through extant_ids on resolution of irods/irods#6921.
	107	# (depending on the nature of the fix we may make it conditional, based on the server --
	108	# if for example in upcoming iRODS 4.2.12 and >=4.3.1 outdated userIDs in R_OBJT_ACCESS
	109	# are guaranteed to be systematically and atomically purged.
	110	extant_ids = set(u[User.id] for u in self.sess.query(User))
	111	rows = [r for r in query_func(target.path) if r[access_column.user_id] in extant_ids]
99	112	userids = set( r[access_column.user_id] for r in rows )
100	113
101	114	user_lookup = { j.id:j for j in users_by_ids(self.sess, userids) }

109	122	acls = [ iRODSAccess ( r[access_column.name],
110	123	target.path,
111	124	user_lookup[r[access_column.user_id]].name,
112		user_lookup[r[access_column.user_id]].zone ) for r in rows ]
	125	user_lookup[r[access_column.user_id]].zone,
	126	user_lookup[r[access_column.user_id]].type,
	127	) for r in rows ]
113	128	return acls
114	129
115	130
116		def set(self, acl, recursive=False, admin=False):
	131	def set(self, acl, recursive=False, admin=False, **kw):
	132
	133	if not kw.pop('suppress_deprecation_warning',False):
	134	warnings.warn('Use of session_obj.permissions is deprecated in v1.1.6',
	135	DeprecationWarning, stacklevel = 2)
	136
117	137	prefix = 'admin:' if admin else ''
118	138
119	139	userName_=acl.user_name
120	140	zone_=acl.user_zone
121	141	if acl.access_name.endswith('inherit'): zone_ = userName_ = ''
122
	142	acl = acl.copy(decanonicalize = True)
123	143	message_body = ModAclRequest(
124	144	recursiveFlag=int(recursive),
125	145	accessLevel='{prefix}{access_name}'.format(prefix=prefix, **vars(acl)),

+43

-25

irods/manager/data_object_manager.py less more

289	289	handle = self.open(arg,_raw_fd_holder=holder,*kw_options)
290	290	return (handle, holder[-1])
291	291
	292	_RESC_flags_for_open = frozenset((
	293	kw.RESC_NAME_KW,
	294	kw.DEST_RESC_NAME_KW, # may be deprecated in the future
	295	kw.RESC_HIER_STR_KW
	296	))
	297
292	298	def open(self, path, mode, create = True, finalize_on_close = True, **options):
293	299	_raw_fd_holder = options.get('_raw_fd_holder',[])
294		if kw.DEST_RESC_NAME_KW not in options:
	300	# If no keywords are used that would influence the server as to the choice of a storage resource,
	301	# then use the default resource in the client configuration.
	302	if self._RESC_flags_for_open.isdisjoint( options.keys() ):
295	303	# Use client-side default resource if available
296	304	try:
297	305	options[kw.DEST_RESC_NAME_KW] = self.sess.default_resource

337	345
338	346	def trim(self, path, **options):
339	347
340		message_body = FileOpenRequest(
341		objPath=path,
342		createMode=0,
343		openFlags=0,
344		offset=0,
345		dataSize=-1,
346		numThreads=self.sess.numThreads,
347		KeyValPair_PI=StringStringMap(options),
348		)
349		message = iRODSMessage('RODS_API_REQ', msg=message_body,
350		int_info=api_number['DATA_OBJ_TRIM_AN'])
351
352		with self.sess.pool.get_connection() as conn:
353		conn.send(message)
354		response = conn.recv()
355
356
357		def unlink(self, path, force=False, **options):
358		if force:
359		options[kw.FORCE_FLAG_KW] = ''
360
361	348	try:
362	349	oprType = options[kw.OPR_TYPE_KW]
363	350	except KeyError:

374	361	KeyValPair_PI=StringStringMap(options),
375	362	)
376	363	message = iRODSMessage('RODS_API_REQ', msg=message_body,
	364	int_info=api_number['DATA_OBJ_TRIM_AN'])
	365
	366	with self.sess.pool.get_connection() as conn:
	367	conn.send(message)
	368	response = conn.recv()
	369
	370
	371	def unlink(self, path, force=False, **options):
	372	if force:
	373	options[kw.FORCE_FLAG_KW] = ''
	374
	375	try:
	376	oprType = options[kw.OPR_TYPE_KW]
	377	except KeyError:
	378	oprType = 0
	379
	380	message_body = FileOpenRequest(
	381	objPath=path,
	382	createMode=0,
	383	openFlags=0,
	384	offset=0,
	385	dataSize=-1,
	386	numThreads=self.sess.numThreads,
	387	oprType=oprType,
	388	KeyValPair_PI=StringStringMap(options),
	389	)
	390	message = iRODSMessage('RODS_API_REQ', msg=message_body,
377	391	int_info=api_number['DATA_OBJ_UNLINK_AN'])
378	392
379	393	with self.sess.pool.get_connection() as conn:

383	397
384	398	def unregister(self, path, **options):
385	399	# https://github.com/irods/irods/blob/4.2.1/lib/api/include/dataObjInpOut.h#L190
386		options[kw.OPR_TYPE_KW] = 26
387
388		self.unlink(path, **options)
	400	options[kw.OPR_TYPE_KW] = 26 # UNREG_OPR: prevents deletion from disk.
	401
	402	# If a replica is targeted, use trim API.
	403	if {kw.RESC_NAME_KW, kw.REPL_NUM_KW} & set(options.keys()):
	404	self.trim(path, **options)
	405	else:
	406	self.unlink(path, **options)
389	407
390	408
391	409	def exists(self, path):

+119

-37

irods/manager/user_manager.py less more

0	0	from __future__ import absolute_import
1	1	import logging
2	2	import os
3
4		from irods.models import User, UserGroup
	3	import warnings
	4
	5	from irods.models import User, Group
5	6	from irods.manager import Manager
6	7	from irods.message import UserAdminRequest, GeneralAdminRequest, iRODSMessage, GetTempPasswordForOtherRequest, GetTempPasswordForOtherOut
7		from irods.exception import UserDoesNotExist, UserGroupDoesNotExist, NoResultFound, CAT_SQL_ERR
	8	from irods.exception import UserDoesNotExist, GroupDoesNotExist, NoResultFound, CAT_SQL_ERR
8	9	from irods.api_number import api_number
9		from irods.user import iRODSUser, iRODSUserGroup
	10	from irods.user import iRODSUser, iRODSGroup, Bad_password_change_parameter
10	11	import irods.password_obfuscation as obf
	12	from .. import MAX_PASSWORD_LENGTH
11	13
12	14	logger = logging.getLogger(__name__)
13
14	15
15	16	class UserManager(Manager):
16	17

25	26	except NoResultFound:
26	27	raise UserDoesNotExist()
27	28	return iRODSUser(self, result)
	29
	30	def create_with_password(self, user_name, password, user_zone = ''):
	31	"""This method can be used by a groupadmin to initialize the password field while creating the new user.
	32	(This is necessary since group administrators may not change the password of an existing user.)
	33	"""
	34
	35	message_body = UserAdminRequest(
	36	"mkuser", user_name, "" if not password else \
	37	obf.obfuscate_new_password_with_key(password, self.sess.pool.account.password)
	38	)
	39	request = iRODSMessage("RODS_API_REQ", msg=message_body,
	40	int_info=api_number['USER_ADMIN_AN'])
	41	with self.sess.pool.get_connection() as conn:
	42	conn.send(request)
	43	response = conn.recv()
	44	logger.debug(response.int_info)
	45	return self.get(user_name, user_zone)
	46
28	47
29	48	def create(self, user_name, user_type, user_zone="", auth_str=""):
30	49	message_body = GeneralAdminRequest(

113	132	the absolute path of an IRODS_AUTHENTICATION_FILE to be altered.
114	133	"""
115	134	with self.sess.pool.get_connection() as conn:
	135
	136	if old_value != self.sess.pool.account.password or not isinstance(new_value, str) \
	137	or not(3 <= len(new_value) <= MAX_PASSWORD_LENGTH - 8):
	138	raise Bad_password_change_parameter
116	139
117	140	hash_new_value = obf.obfuscate_new_password(new_value, old_value, conn.client_signature)
118	141

175	198	logger.debug(response.int_info)
176	199
177	200
178		class UserGroupManager(UserManager):
	201	class GroupManager(UserManager):
179	202
180	203	def get(self, name, user_zone=""):
181		query = self.sess.query(UserGroup).filter(UserGroup.name == name)
	204	query = self.sess.query(Group).filter(Group.name == name)
182	205
183	206	try:
184	207	result = query.one()
185	208	except NoResultFound:
186		raise UserGroupDoesNotExist()
187		return iRODSUserGroup(self, result)
188
189		def create(self, name, user_type='rodsgroup', user_zone="", auth_str=""):
190		message_body = GeneralAdminRequest(
191		"add",
192		"user",
193		name,
194		user_type,
195		"",
196		""
197		)
198		request = iRODSMessage("RODS_API_REQ", msg=message_body,
199		int_info=api_number['GENERAL_ADMIN_AN'])
	209	raise GroupDoesNotExist()
	210	return iRODSGroup(self, result)
	211
	212	def _api_info(self, group_admin_flag):
	213	"""
	214	If group_admin_flag is: then use UserAdminRequest API:
	215	--------------------------- ------------------------------
	216	True always
	217	False (user_groups default) never
	218	None (groups default) when user type is groupadmin
	219	"""
	220
	221	sess = self.sess
	222	if group_admin_flag or (group_admin_flag is not False and sess.users.get(sess.username).type == 'groupadmin'):
	223	return (UserAdminRequest, 'USER_ADMIN_AN')
	224	return (GeneralAdminRequest, 'GENERAL_ADMIN_AN')
	225
	226	def create(self, name,
	227	user_type = 'rodsgroup',
	228	user_zone = "",
	229	auth_str = "",
	230	group_admin = False, **options):
	231
	232	if not options.pop('suppress_deprecation_warning', False):
	233	warnings.warn('Use of session.user_groups is deprecated in v1.1.7 - prefer session.groups',
	234	DeprecationWarning, stacklevel = 2)
	235
	236	(MessageClass, api_key) = self._api_info(group_admin)
	237	api_to_use = api_number[api_key]
	238
	239	if MessageClass is UserAdminRequest:
	240	message_body = MessageClass(
	241	"mkgroup",
	242	name,
	243	user_type,
	244	user_zone
	245	)
	246	else:
	247	message_body = MessageClass(
	248	"add",
	249	"user",
	250	name,
	251	user_type,
	252	"",
	253	""
	254	)
	255	request = iRODSMessage("RODS_API_REQ", msg=message_body,
	256	int_info=api_to_use)
200	257	with self.sess.pool.get_connection() as conn:
201	258	conn.send(request)
202	259	response = conn.recv()

205	262
206	263	def getmembers(self, name):
207	264	results = self.sess.query(User).filter(
208		User.type != 'rodsgroup', UserGroup.name == name)
	265	User.type != 'rodsgroup', Group.name == name)
209	266	return [iRODSUser(self, row) for row in results]
210	267
211		def addmember(self, group_name, user_name, user_zone=""):
212		message_body = GeneralAdminRequest(
	268	def addmember(self, group_name,
	269	user_name,
	270	user_zone = "",
	271	group_admin = False, **options):
	272
	273	if not options.pop('suppress_deprecation_warning', False):
	274	warnings.warn('Use of session.user_groups is deprecated in v1.1.7 - prefer session.groups',
	275	DeprecationWarning, stacklevel = 2)
	276
	277	(MessageClass, api_key) = self._api_info(group_admin)
	278
	279	message_body = MessageClass(
213	280	"modify",
214	281	"group",
215	282	group_name,

218	285	user_zone
219	286	)
220	287	request = iRODSMessage("RODS_API_REQ", msg=message_body,
221		int_info=api_number['GENERAL_ADMIN_AN'])
222		with self.sess.pool.get_connection() as conn:
223		conn.send(request)
224		response = conn.recv()
225		logger.debug(response.int_info)
226
227		def removemember(self, group_name, user_name, user_zone=""):
228		message_body = GeneralAdminRequest(
	288	int_info=api_number[api_key])
	289	with self.sess.pool.get_connection() as conn:
	290	conn.send(request)
	291	response = conn.recv()
	292	logger.debug(response.int_info)
	293
	294	def removemember(self,
	295	group_name,
	296	user_name,
	297	user_zone = "",
	298	group_admin = False, **options):
	299
	300	if not options.pop('suppress_deprecation_warning', False):
	301	warnings.warn('Use of session.user_groups is deprecated in v1.1.7 - prefer session.groups',
	302	DeprecationWarning, stacklevel = 2)
	303
	304	(MessageClass, api_key) = self._api_info(group_admin)
	305
	306	message_body = MessageClass(
229	307	"modify",
230	308	"group",
231	309	group_name,

234	312	user_zone
235	313	)
236	314	request = iRODSMessage("RODS_API_REQ", msg=message_body,
237		int_info=api_number['GENERAL_ADMIN_AN'])
238		with self.sess.pool.get_connection() as conn:
239		conn.send(request)
240		response = conn.recv()
241		logger.debug(response.int_info)
	315	int_info=api_number[api_key])
	316	with self.sess.pool.get_connection() as conn:
	317	conn.send(request)
	318	response = conn.recv()
	319	logger.debug(response.int_info)
	320
	321
	322	# NOTE: Everything of the form UserGroup is deprecated.
	323	UserGroupManager = GroupManager

+30

-4

irods/message/__init__.py less more

18	18	IntegerProperty, LongProperty, ArrayProperty,
19	19	SubmessageProperty)
20	20
	21	class Bad_AVU_Field(Exception):
	22	pass
	23
21	24	_TUPLE_LIKE_TYPES = (tuple, list)
	25
22	26
23	27	def _qxml_server_version( var ):
24	28	val = os.environ.get( var, '()' )

141	145
142	146	logger = logging.getLogger(__name__)
143	147
144		IRODS_VERSION = (4, 3, 0, 'd')
	148	IRODS_VERSION = (4, 3, 1, 'd')
145	149
146	150	try:
147	151	# Python 2

150	154	# Python 3
151	155	UNICODE = str
152	156
	157	_METADATA_FIELD_TYPES = {str,UNICODE,bytes}
153	158
154	159
155	160	# Necessary for older python (<3.7):

669	674
670	675	def __init__(self, args, *metadata_opts):
671	676	super(MetadataRequest, self).__init__()
672		for i in range(len(args)):
673		if args[i]:
674		setattr(self, 'arg%d' % i, args[i])
	677
	678	NoneType = type(None)
	679	def field_name(i) : return ("attribute", "value", "unit")[i-3]
	680
	681	# We now enforce these requirements of the scalars (ie. AVU fields) submitted to the metadata call:
	682	# * All fields must each be of type str, except that the units field can be the None object.
	683	# * Attribute and Value must be of type str (a Python string) as well as nonzero length.
	684	for i,arg in enumerate(args):
	685	error = None
	686
	687	# Raise usage error if any of the AVU fields (args 3 to 5 inclusive) do not meet the above constraints.
	688	if i in (3,4,5):
	689	if type(arg) not in (_METADATA_FIELD_TYPES if i<5
	690	else {NoneType,}\|_METADATA_FIELD_TYPES):
	691	error = Bad_AVU_Field("AVU %s (%r) has incorrect type. AVU fields must be strings, except for units, which could be None." % (field_name(i),arg))
	692	elif i<5 and not(arg):
	693	error = Bad_AVU_Field("AVU %s (%r) is zero-length." % (field_name(i), arg))
	694	if error is not None:
	695	raise error
	696
	697	# If there is no error, set the attribute in the request message.
	698	if arg not in {None, b'', ''}:
	699	setattr(self, 'arg%d' % i, arg)
	700
675	701	self.KeyValPair_PI = StringStringMap(metadata_opts)
676	702
677	703	arg0 = StringProperty()

+7

-1

irods/meta.py less more

0
	0	import six
1	1
2	2	class iRODSMeta(object):
3	3

94	94	"""
95	95	Returns a list of iRODSMeta associated with a given key
96	96	"""
	97	if six.PY2:
	98	if isinstance(key, unicode):
	99	key = key.encode('utf8')
	100	else:
	101	if isinstance(key, bytes):
	102	key = key.decode('utf8')
97	103	if not isinstance(key, str):
98	104	raise TypeError
99	105	return [m for m in self._meta if m.name == key]

+4

-1

irods/models.py less more

65	65	zone = Column(String, 'COL_COLL_USER_ZONE', 1301)
66	66
67	67
68		class UserGroup(Model):
	68	class Group(Model):
69	69	id = Column(Integer, 'USER_GROUP_ID', 900)
70	70	name = Column(String, 'USER_GROUP_NAME', 901)
	71
	72	# The UserGroup model-class is now renamed Group, but we'll keep the deprecated name around for now.
	73	UserGroup = Group
71	74
72	75
73	76	class Resource(Model):

+36

-2

irods/password_obfuscation.py less more

40	40	default_scramble_prefix = '.E_'
41	41	v2_prefix = 'A.ObfV2'
42	42
	43
	44	def str_to_int(input_string):
	45	"""Convert `input_string` into an integer.
	46
	47	Parameters
	48	----------
	49	input_string : str
	50	Value to convert.
	51
	52	Returns
	53	-------
	54	int
	55	Converted value.
	56	"""
	57	return sum((ord(char) for char in input_string))
	58
	59
43	60	#Decode a password from a .irodsA file
44	61	def decode(s, uid=None):
45	62	#This value lets us know which seq value to use

55	72
56	73	#The uid is used as a salt.
57	74	if uid is None:
58		uid = os.getuid()
	75	try:
	76	uid = os.getuid()
	77	except AttributeError:
	78	# Spoof UID for Non-POSIX
	79	uid = str_to_int(os.getlogin())
59	80
60	81	#The first byte is a dot, the next five are literally irrelevant
61	82	#garbage, and we already used the seventh one. The string to decode

100	121
101	122	#The uid is used as a salt.
102	123	if uid is None:
103		uid = os.getuid()
	124	try:
	125	uid = os.getuid()
	126	except AttributeError:
	127	# Spoof UID for Non-POSIX
	128	uid = str_to_int(os.getlogin())
104	129
105	130	#This value lets us know which seq value to use
106	131	#Referred to as "rval" in the C code

262	287
263	288	return scramble(to_scramble, key=hashed_key, scramble_prefix='', block_chaining=True)
264	289
	290	def obfuscate_new_password_with_key(new_password, obfuscation_key):
	291	lcopy = MAX_PASSWORD_LENGTH - 10 - len(new_password)
	292	new_password = new_password[:MAX_PASSWORD_LENGTH]
	293	if lcopy > 15:
	294	# https://github.com/irods/irods/blob/4.2.1/plugins/database/src/db_plugin.cpp#L1094-L1095
	295	padding = '1gCBizHWbwIYyWLoysGzTe6SyzqFKMniZX05faZHWAwQKXf6Fs'
	296	new_password += padding[:lcopy]
	297	return scramble(new_password, obfuscation_key, scramble_prefix='')
	298
265	299	# port of https://github.com/irods/irods_client_icommands/blob/4.2.1/src/iadmin.cpp#L878-L930
266	300	def obfuscate_new_password(new, old, signature):
267	301	pwd_len = len(new)

+2

-0

irods/pool.py less more

75	75	logger.debug("No connection found in idle set. Created a new connection with id: {}".format(id(conn)))
76	76
77	77	self.active.add(conn)
	78
78	79	logger.debug("Adding connection with id {} to active set".format(id(conn)))
79	80
80	81	logger.debug('num active: {}'.format(len(self.active)))
81	82	logger.debug('num idle: {}'.format(len(self.idle)))
	83
82	84	return conn
83	85
84	86	def release_connection(self, conn, destroy=False):

+51

-0

irods/resource.py less more

6	6	class iRODSResource(object):
7	7
8	8	def __init__(self, manager, result=None):
	9	self._hierarchy_string = ''
	10	self._parent_name = ''
	11	self._parent_id = ''
9	12	'''
10	13	self.id = result[Resource.id]
11	14	self.name = result[Resource.name]

36	39	pass
37	40
38	41	self._meta = None
	42
	43
	44	## Cached properties to expose parent id or name regardless whether the DB model is iRODS 4.1- or 4.2+
	45
	46	@property
	47	def parent_id(self):
	48	if self.parent is None:
	49	return None
	50	if self._parent_id == '':
	51	sess = self.manager.sess
	52	if sess.server_version >= (4, 2, 0):
	53	self._parent_id = self.parent
	54	else:
	55	self._parent_id = sess.query(Resource).filter(Resource.name == self.parent).one()[Resource.id]
	56	return int(self._parent_id)
	57
	58
	59	@property
	60	def parent_name(self):
	61	if self.parent is None:
	62	return None
	63	if self._parent_name == '':
	64	sess = self.manager.sess
	65	if sess.server_version < (4, 2, 0):
	66	self._parent_name = self.parent
	67	else:
	68	self._parent_name = sess.query(Resource).filter(Resource.id == self.parent).one()[Resource.name]
	69	return self._parent_name
	70
	71	## Cached property to expose resource hierarchy string
	72
	73	@property
	74	def hierarchy_string(self):
	75	if self._hierarchy_string == '':
	76	self._hierarchy_string = ';'.join(r.name for r in self.hierarchy_as_list_of_resource_objects())
	77	return self._hierarchy_string
	78
	79	## Retrieve chain of parent objects to top level parent
	80
	81	def hierarchy_as_list_of_resource_objects(self):
	82	trace_to_root = [self]
	83	sess = self.manager.sess
	84	r = self.parent_id
	85	while r is not None:
	86	parent = sess.query(Resource).filter(Resource.id == r).one()
	87	trace_to_root.append(iRODSResource(self.manager, parent))
	88	r = trace_to_root[-1].parent_id
	89	return list(reversed(trace_to_root))
39	90
40	91	@property
41	92	def metadata(self):

+106

-4

irods/session.py less more

0	0	from __future__ import absolute_import
	1	import atexit
1	2	import os
2	3	import ast
3	4	import json

10	11	from irods.manager.data_object_manager import DataObjectManager
11	12	from irods.manager.metadata_manager import MetadataManager
12	13	from irods.manager.access_manager import AccessManager
13		from irods.manager.user_manager import UserManager, UserGroupManager
	14	from irods.manager.user_manager import UserManager, GroupManager
14	15	from irods.manager.resource_manager import ResourceManager
15	16	from irods.manager.zone_manager import ZoneManager
16	17	from irods.exception import NetworkException
17	18	from irods.password_obfuscation import decode
18	19	from irods import NATIVE_AUTH_SCHEME, PAM_AUTH_SCHEME
	20	import threading
	21	import weakref
	22	from . import DEFAULT_CONNECTION_TIMEOUT
	23
	24	_sessions = None
	25	_sessions_lock = threading.Lock()
	26
	27	def _cleanup_remaining_sessions():
	28	for ses in _sessions.copy():
	29	ses.cleanup() # internally modifies _sessions
	30
	31	def _weakly_reference(ses):
	32	global _sessions
	33	try:
	34	if _sessions is None:
	35	with _sessions_lock:
	36	do_register = (_sessions is None)
	37	if do_register:
	38	_sessions = weakref.WeakKeyDictionary()
	39	atexit.register(_cleanup_remaining_sessions)
	40	finally:
	41	_sessions[ses] = None
19	42
20	43	logger = logging.getLogger(__name__)
21	44

31	54	def auth_file (self):
32	55	return self._auth_file
33	56
34		def __init__(self, configure=True, **kwargs):
	57	# session.acls will act identically to session.permissions, except its `get'
	58	# method has a default parameter of report_raw_acls = True, so it enumerates
	59	# ACLs exactly in the manner of "ils -A".
	60
	61	@property
	62	def available_permissions(self):
	63	from irods.access import (iRODSAccess,_iRODSAccess_pre_4_3_0)
	64	try:
	65	self.__access
	66	except AttributeError:
	67	self.__access = _iRODSAccess_pre_4_3_0 if self.server_version < (4,3) else iRODSAccess
	68	return self.__access
	69
	70	@property
	71	def groups(self):
	72	class _GroupManager(self.user_groups.__class__):
	73
	74	def create(self, name,
	75	group_admin = None): # NB new default (see user_groups manager and i/f, with False as default)
	76
	77	user_type = 'rodsgroup' # These are no longer parameters in the new interface, as they have no reason to vary.
	78	user_zone = "" # Groups (1) are always of type 'rodsgroup', (2) always belong to the local zone, and
	79	auth_str = "" # (3) do not authenticate.
	80
	81	return super(_GroupManager, self).create(name,
	82	user_type,
	83	user_zone,
	84	auth_str,
	85	group_admin,
	86	suppress_deprecation_warning = True)
	87
	88	def addmember(self, group_name,
	89	user_name,
	90	user_zone = "",
	91	group_admin = None):
	92
	93	return super(_GroupManager, self).addmember(group_name,
	94	user_name,
	95	user_zone,
	96	group_admin,
	97	suppress_deprecation_warning = True)
	98
	99	def removemember(self, group_name,
	100	user_name,
	101	user_zone = "",
	102	group_admin = None):
	103
	104	return super(_GroupManager, self).removemember(group_name,
	105	user_name,
	106	user_zone,
	107	group_admin,
	108	suppress_deprecation_warning = True)
	109
	110	_groups = getattr(self,'_groups',None)
	111	if not _groups:
	112	_groups = self._groups = _GroupManager(self.user_groups.sess)
	113	return self._groups
	114
	115	@property
	116	def acls(self):
	117	class ACLs(self.permissions.__class__):
	118	def set(self, acl, recursive=False, admin=False, **kw):
	119	kw['suppress_deprecation_warning'] = True
	120	return super(ACLs, self).set(acl, recursive=recursive, admin=admin, **kw)
	121	def get(self, target, **kw):
	122	kw['suppress_deprecation_warning'] = True
	123	return super(ACLs,self).get(target, report_raw_acls = True, **kw)
	124	_acls = getattr(self,'_acls',None)
	125	if not _acls: _acls = self._acls = ACLs(self.permissions.sess)
	126	return _acls
	127
	128	def __init__(self, configure = True, auto_cleanup = True, **kwargs):
35	129	self.pool = None
36	130	self.numThreads = 0
37	131	self._env_file = ''
38	132	self._auth_file = ''
39	133	self.do_configure = (kwargs if configure else {})
	134	self._cached_connection_timeout = kwargs.pop('connection_timeout', DEFAULT_CONNECTION_TIMEOUT)
40	135	self.__configured = None
41	136	if configure:
42	137	self.__configured = self.configure(**kwargs)

46	141	self.metadata = MetadataManager(self)
47	142	self.permissions = AccessManager(self)
48	143	self.users = UserManager(self)
49		self.user_groups = UserGroupManager(self)
	144	self.user_groups = GroupManager(self)
50	145	self.resources = ResourceManager(self)
51	146	self.zones = ZoneManager(self)
	147
	148	if auto_cleanup:
	149	_weakly_reference(self)
52	150
53	151	def __enter__(self):
54	152	return self

135	233	connection_refresh_time = self.get_connection_refresh_time(**kwargs)
136	234	logger.debug("In iRODSSession's configure(). connection_refresh_time set to {}".format(connection_refresh_time))
137	235	self.pool = Pool(account, application_name=kwargs.pop('application_name',''), connection_refresh_time=connection_refresh_time)
	236	conn_timeout = getattr(self,'_cached_connection_timeout',None)
	237	self.pool.connection_timeout = conn_timeout
138	238	return account
139	239
140	240	def query(self, *args):

197	297
198	298	@connection_timeout.setter
199	299	def connection_timeout(self, seconds):
200		self.pool.connection_timeout = seconds
	300	self._cached_connection_timeout = seconds
	301	if seconds is not None:
	302	self.pool.connection_timeout = seconds
201	303
202	304	@staticmethod
203	305	def get_irods_password_file():

+136

-21

irods/test/access_test.py less more

0	0	#! /usr/bin/env python
1	1	from __future__ import absolute_import
	2
2	3	import os
3	4	import sys
4	5	import unittest
	6
5	7	from irods.access import iRODSAccess
	8	from irods.collection import iRODSCollection
	9	from irods.column import In, Like
	10	from irods.exception import UserDoesNotExist
	11	from irods.models import User,Collection,DataObject
6	12	from irods.user import iRODSUser
7	13	from irods.session import iRODSSession
8		from irods.models import User,Collection,DataObject
9		from irods.collection import iRODSCollection
10	14	import irods.test.helpers as helpers
11		from irods.column import In, Like
12	15
13	16
14	17	class TestAccess(unittest.TestCase):

48	51
49	52	# test exception
50	53	with self.assertRaises(TypeError):
51		self.sess.permissions.get(filename)
	54	self.sess.acls.get(filename)
52	55
53	56	# get object's ACLs
54	57	# only one for now, the owner's own access
55		acl = self.sess.permissions.get(obj)[0]
	58	acl = self.sess.acls.get(obj)[0]
56	59
57	60	# check values
58	61	self.assertEqual(acl.access_name, 'own')

61	64
62	65	# check repr()
63	66	self.assertEqual(
64		repr(acl), "<iRODSAccess own {path} {name} {zone}>".format(path=path, **vars(user)))
	67	repr(acl), "<iRODSAccess own {path} {name}({type}) {zone}>".format(path=path, **vars(user)))
65	68
66	69	# remove object
67	70	self.sess.data_objects.unlink(path)

70	73	def test_set_inherit_acl(self):
71	74
72	75	acl1 = iRODSAccess('inherit', self.coll_path)
73		self.sess.permissions.set(acl1)
	76	self.sess.acls.set(acl1)
74	77	c = self.sess.collections.get(self.coll_path)
75	78	self.assertTrue(c.inheritance)
76	79
77	80	acl2 = iRODSAccess('noinherit', self.coll_path)
78		self.sess.permissions.set(acl2)
	81	self.sess.acls.set(acl2)
79	82	c = self.sess.collections.get(self.coll_path)
80	83	self.assertFalse(c.inheritance)
	84
	85	def test_available_permissions__420_422(self):
	86	# Cycle through access levels (strings available via session.available_permissions) and test, with
	87	# a string compare, that the "set" access level matches the "get" access level.
	88	user = data = None
	89	try:
	90	user = self.sess.users.create('bob','rodsuser')
	91	data = self.sess.data_objects.create('{}/obj_422'.format(helpers.home_collection(self.sess)))
	92	permission_strings = self.sess.available_permissions.keys()
	93	for perm in permission_strings:
	94	access = iRODSAccess(perm, data.path, 'bob')
	95	self.sess.acls.set( access )
	96	a = [acl for acl in self.sess.acls.get( data ) if acl.user_name == 'bob']
	97	if perm == 'null':
	98	self.assertEqual(len(a),0)
	99	else:
	100	self.assertEqual(len(a),1)
	101	self.assertEqual(access.access_name,a[0].access_name)
	102	finally:
	103	if user: user.remove()
	104	if data: data.unlink(force=True)
81	105
82	106	def test_set_inherit_and_test_sub_objects (self):
83	107	DEPTH = 3

93	117	acl_inherit = iRODSAccess('inherit', deepcoll.path)
94	118	acl_read = iRODSAccess('read', deepcoll.path, 'bob')
95	119
96		self.sess.permissions.set(acl_read)
97		self.sess.permissions.set(acl_inherit)
	120	self.sess.acls.set(acl_read)
	121	self.sess.acls.set(acl_inherit)
98	122
99	123	# create one new object and one new collection after ACL's are applied
100	124	new_object_path = test_coll_path + "/my_data_obj"

144	168	test_coll_path = self.coll_path + "/test"
145	169	deepcoll = helpers.make_deep_collection(self.sess, test_coll_path, depth=DEPTH, objects_per_level=2)
146	170	acl1 = iRODSAccess('inherit', deepcoll.path)
147		self.sess.permissions.set( acl1, recursive = recursionTruth )
	171	self.sess.acls.set( acl1, recursive = recursionTruth )
148	172	test_subcolls = set( iRODSCollection(self.sess.collections,_)
149	173	for _ in self.sess.query(Collection).filter(Like(Collection.name, deepcoll.path + "/%")) )
150	174

177	201
178	202	# set permission to write
179	203	acl1 = iRODSAccess('write', path, user.name, user.zone)
180		self.sess.permissions.set(acl1)
	204	self.sess.acls.set(acl1)
181	205
182	206	# get object's ACLs
183		acl = self.sess.permissions.get(obj)[0] # owner's write access
	207	acl = self.sess.acls.get(obj)[0] # owner's write access
184	208
185	209	# check values
186	210	self.assertEqual(acl.access_name, self.mapping['write'])

189	213
190	214	# reset permission to own
191	215	acl1 = iRODSAccess('own', path, user.name, user.zone)
192		self.sess.permissions.set(acl1)
	216	self.sess.acls.set(acl1, admin = True)
193	217
194	218	# remove object
195	219	self.sess.data_objects.unlink(path)

203	227
204	228	# set permission to write
205	229	acl1 = iRODSAccess('write', coll.path, user.name, user.zone)
206		self.sess.permissions.set(acl1)
	230	self.sess.acls.set(acl1)
207	231
208	232	# get collection's ACLs
209		acl = self.sess.permissions.get(coll)[0] # owner's write access
	233	acl = self.sess.acls.get(coll)[0] # owner's write access
210	234
211	235	# check values
212	236	self.assertEqual(acl.access_name, self.mapping['write'])

215	239
216	240	# reset permission to own
217	241	acl1 = iRODSAccess('own', coll.path, user.name, user.zone)
218		self.sess.permissions.set(acl1)
	242	self.sess.acls.set(acl1, admin = True)
219	243
220	244	def perms_lists_symm_diff ( self, a_iter, b_iter ):
221	245	fields = lambda perm: (self.mapping[perm.access_name], perm.user_name, perm.user_zone)

227	251	data = helpers.make_object(self.sess,"/".join((self.coll_path,"test_obj")))
228	252	eg = eu = fg = fu = None
229	253	try:
230		eg = self.sess.user_groups.create ('egrp')
	254	eg = self.sess.groups.create ('egrp')
231	255	eu = self.sess.users.create ('edith','rodsuser')
232	256	eg.addmember(eu.name,eu.zone)
233		fg = self.sess.user_groups.create ('fgrp')
	257	fg = self.sess.groups.create ('fgrp')
234	258	fu = self.sess.users.create ('frank','rodsuser')
235	259	fg.addmember(fu.name,fu.zone)
236	260	my_ownership = set([('own', self.sess.username, self.sess.zone)])

238	262	perms1data = [ iRODSAccess ('write',self.coll_path, eg.name, self.sess.zone),
239	263	iRODSAccess ('read', self.coll_path, fu.name, self.sess.zone)
240	264	]
241		for perm in perms1data: self.sess.permissions.set ( perm )
	265	for perm in perms1data: self.sess.acls.set ( perm )
242	266	p1 = self.sess.permissions.get ( self.coll, report_raw_acls = True)
243	267	self.assertEqual(self.perms_lists_symm_diff( perms1data, p1 ), my_ownership)
244	268	#--data object--
245	269	perms2data = [ iRODSAccess ('write',data.path, fg.name, self.sess.zone),
246	270	iRODSAccess ('read', data.path, eu.name, self.sess.zone)
247	271	]
248		for perm in perms2data: self.sess.permissions.set ( perm )
	272	for perm in perms2data: self.sess.acls.set ( perm )
249	273	p2 = self.sess.permissions.get ( data, report_raw_acls = True)
250	274	self.assertEqual(self.perms_lists_symm_diff( perms2data, p2 ), my_ownership)
251	275	finally:

254	278	for row in self.sess.query(User).filter(In(User.id, ids_for_delete)) ]:
255	279	u.remove()
256	280
	281	def test_ses_acls_data_and_collection_395_396(self):
	282	ses = self.sess
	283	try:
	284	# Create rodsusers alice and bob, and make bob a member of the 'team' group.
	285
	286	self.alice = ses.users.create('alice','rodsuser')
	287	self.bob = ses.users.create('bob','rodsuser')
	288	self.team = ses.groups.create('team')
	289	self.team.addmember('bob')
	290	ses.users.modify('bob', 'password', 'bpass')
	291
	292	# Create a collection and a data object.
	293
	294	home = helpers.home_collection(ses)
	295	self.coll_395 = ses.collections.create(home + "/coll-395")
	296	self.data = ses.data_objects.create(self.coll_395.path +"/data")
	297	with self.data.open('w') as f: f.write(b'contents-395')
	298
	299	# Make assertions for both filesystem object types (collection and data):
	300
	301	for obj in self.coll_395, self.data:
	302
	303	# Add ACLs
	304	for access in iRODSAccess('read',obj.path,'team'), iRODSAccess('write',obj.path,'alice'):
	305	ses.acls.set(access)
	306
	307	ACLs = ses.acls.get(obj)
	308
	309	# Assert that we can detect alice's write permissions.
	310	self.assertEqual(1, len([ac for ac in ACLs if (ac.access_name,ac.user_name)
	311	== (self.mapping['write'],'alice')]))
	312
	313	# Test that the 'team' ACL is listed as a rodsgroup ...
	314
	315	team_acl = [ac for ac in ACLs if ac.user_name == 'team']
	316	self.assertEqual(1, len(team_acl))
	317	self.assertEqual(team_acl[0].user_type,
	318	'rodsgroup')
	319
	320	# ... and also that 'bob' (a 'team' member) is not listed explicitly.
	321	self.assertEqual(0, len([ac for ac in ACLs if ac.user_name == 'bob']))
	322
	323	# verify that bob can access the data object as a member of team.
	324	with iRODSSession( host=ses.host,
	325	user='bob',
	326	port=ses.port,
	327	zone=ses.zone,
	328	password = 'bpass') as bob:
	329	self.assertTrue( bob.data_objects.open(self.data.path,'r').read(), b'contents-395' )
	330
	331	finally:
	332	self.coll_395.remove(recurse = True,force = True)
	333	self.bob.remove()
	334	self.alice.remove()
	335	self.team.remove()
	336
	337	def test_removed_user_does_not_affect_raw_ACL_queries__issue_431(self):
	338	user_name = "testuser"
	339	session = self.sess
	340	try:
	341	# Create user and collection.
	342	user = session.users.create(user_name, 'rodsuser')
	343	coll_path = "/{0.zone}/home/test".format(session)
	344	coll = session.collections.create(coll_path)
	345
	346	# Give user access to collection.
	347	access = iRODSAccess('read', coll.path, user.name)
	348	session.acls.set(access)
	349
	350	# We can get permissions from collection, and the test user's entry is there.
	351	perms = session.acls.get(coll)
	352	self.assertTrue(any(p for p in perms if p.user_name == user_name))
	353
	354	# Now we remove the user and try again.
	355	user.remove()
	356
	357	# The following line threw a KeyError prior to the issue #431 fix,
	358	# as already-deleted users' IDs were being returned in the raw ACL queries.
	359	# It appears iRODS as of 4.2.11 and 4.3.0 does not purge R_OBJT_ACCESS of old
	360	# user IDs. (See: https://github.com/irods/irods/issues/6921)
	361	perms = session.acls.get(coll)
	362
	363	# As an extra test, make sure the removed user is gone from the list.
	364	self.assertFalse(any(p for p in perms if p.user_name == user_name))
	365	finally:
	366	try:
	367	u = session.users.get(user_name)
	368	except UserDoesNotExist:
	369	pass
	370	else:
	371	u.remove()
257	372
258	373	if __name__ == '__main__':
259	374	# let the tests find the parent irods lib

+38

-1

irods/test/admin_test.py less more

3	3	import sys
4	4	import unittest
5	5	from irods.models import User
6		from irods.exception import UserDoesNotExist, ResourceDoesNotExist
	6	from irods.exception import UserDoesNotExist, ResourceDoesNotExist, SYS_NO_API_PRIV
7	7	from irods.session import iRODSSession
8	8	from irods.resource import iRODSResource
9	9	import irods.test.helpers as helpers

70	70	# user should be gone
71	71	with self.assertRaises(UserDoesNotExist):
72	72	self.sess.users.get(self.new_user_name, self.sess.zone)
	73
	74
	75	def test_groupadmin_creates_group_and_unable_to_delete_group__374(self):
	76	# Have user with groupadmin
	77	user = self.sess.users.create(self.new_user_name, 'groupadmin')
	78	self.assertEqual(user.name, self.new_user_name)
	79
	80	# Have a password for login
	81	user_password = 'bpass'
	82	user.modify('password', user_password)
	83
	84	group_name = 'coolgroup'
	85
	86	# Create session as user
	87	with iRODSSession(port=self.sess.port, zone=self.sess.zone, host=self.sess.host,
	88	user=self.new_user_name, password=user_password) as bobby:
	89	# Create group
	90	group = bobby.groups.create(group_name, group_admin=True)
	91	self.assertEqual(group.name, group_name)
	92
	93	# groupadmin cannot remove groups!
	94	with self.assertRaises(SYS_NO_API_PRIV):
	95	group.remove()
	96
	97	# Only an admin can remove groups!
	98	self.sess.groups.get(group_name).remove()
	99	user.remove()
	100
	101
	102	def test_admin_creates_and_deletes_group__374(self):
	103	group_name = 'evencoolergroup'
	104
	105	# Create and remove group
	106	group = self.sess.groups.create(group_name)
	107	self.assertEqual(group.name, group_name)
	108
	109	group.remove()
73	110
74	111
75	112	def test_modify_user_type(self):

+55

-0

irods/test/connection_test.py less more

1	1	from __future__ import absolute_import
2	2	import os
3	3	import sys
	4	import tempfile
4	5	import unittest
5	6	from irods.exception import NetworkException
6	7	import irods.test.helpers as helpers

29	30	conn.release(destroy=True)
30	31
31	32	def test_failed_connection(self):
	33	# Make sure no connections are cached in self.sess.pool.idle to be grabbed by get_connection().
	34	# (Necessary after #418 fix; make_session() can probe server_version, which then leaves an idle conn.)
	35	self.sess.cleanup()
32	36	# mess with the account's port
33	37	saved_port = self.sess.port
34	38	self.sess.pool.account.port = 6666

61	65	with self.assertRaises(NetworkException):
62	66	conn.reply(0)
63	67
	68	def test_that_connection_timeout_works__issue_377(self):
	69	sess = self.sess
	70	h = helpers.home_collection(sess)
	71	logical_path = h + '/issue_377_test.file_timeout_test_on_chksum'
	72	rand = os.urandom(1024)*64
	73	obj = local_file = None
	74	try:
	75	# Create a large file.
	76	size = 1024*2 100
	77	with tempfile.NamedTemporaryFile(delete = False) as local_file:
	78	while local_file.tell() < size:
	79	local_file.write(rand)
	80	obj = sess.data_objects.put(local_file.name, logical_path, return_data_object = True)
	81
	82	# Set a very short socket timeout and remove all pre-existing socket connections.
	83	# This forces a new connection to be made for any ensuing connections to the iRODS server.
	84
	85	sess.connection_timeout = timeout = 0.01
	86	sess.cleanup()
	87
	88	# Make sure the newly formed connection pool inherits the timeout value.
	89	self.assertAlmostEqual(sess.pool.connection_timeout, timeout)
	90
	91	# Perform a time-consuming operation in the server (ie. computing the checksum of a
	92	# large data object) during which the socket will time out.
	93	with self.assertRaises(NetworkException):
	94	obj.chksum()
	95	finally:
	96	# Set the connection pool's socket timeout interval back to default, and clean up.
	97	sess.connection_timeout = None
	98	sess.cleanup()
	99	obj.unlink(force = True)
	100	if local_file:
	101	os.unlink(local_file.name)
	102
	103	def assert_timeout_value_propagated_to_socket(self, session, timeout_value):
	104	session.collections.get(helpers.home_collection(session))
	105	conn = next(iter(session.pool.idle))
	106	self.assertEqual(conn.socket.gettimeout(), timeout_value)
	107
	108	def test_connection_timeout_parameter_in_session_init__issue_377(self):
	109	timeout = 1.0
	110	sess = helpers.make_session(connection_timeout = timeout)
	111	self.assert_timeout_value_propagated_to_socket(sess, timeout)
	112
	113	def test_assigning_session_connection_timeout__issue_377(self):
	114	sess = self.sess
	115	for timeout in (999999, None):
	116	sess.connection_timeout = timeout
	117	sess.cleanup()
	118	self.assert_timeout_value_propagated_to_socket(sess, timeout)
64	119
65	120	if __name__ == '__main__':
66	121	# let the tests find the parent irods lib

+157

-3

irods/test/data_obj_test.py less more

0	0	#! /usr/bin/env python
1	1	from __future__ import absolute_import
2	2	import os
	3	import stat
3	4	import sys
4	5	import socket
5	6	import json

31	32	import irods.parallel
32	33	from irods.manager.data_object_manager import Server_Checksum_Warning
33	34
34
35		def make_ufs_resc_in_tmpdir(session, base_name, allow_local = False):
	35	def make_ufs_resc_in_tmpdir(session, base_name, allow_local = False, client_vault_mode = (True,)):
	36	# Parameters
	37	# ----------
	38	# base_name - The name for the resource, as well as the root directory of the Vault. Use something random and unlikely to collide.
	39	# allow_local - Whether to allow the resource's vault to be located under a non-shared ie. "/tmp" style directory.
	40	# client_vault_mode - A tuple of (client_mkdir[, mode_OR_mask]):
	41	# client_mkdir - whether to call mkdir on the vault-path from the client side, and ...
	42	# mode_OR_mask - if so, what mode bits to be OR'ed into the permission of the vault path after creation
	43	# (A typical value might be : 0o777 \| stat.S_ISGID, to guarantee iRODS has permissions on the vault)
36	44	tmpdir = helpers.irods_shared_tmp_dir()
37	45	if not tmpdir and allow_local:
38	46	tmpdir = os.getenv('TMPDIR') or '/tmp'
39	47	if not tmpdir:
40	48	raise RuntimeError("Must have filesystem path shareable with server.")
	49
41	50	full_phys_dir = os.path.join(tmpdir,base_name)
42		if not os.path.exists(full_phys_dir): os.mkdir(full_phys_dir)
	51
	52	if client_vault_mode[0]:
	53	if not os.path.exists(full_phys_dir):
	54	os.mkdir(full_phys_dir)
	55	guarantee_mode_bits = tuple(client_vault_mode[1:])
	56	if guarantee_mode_bits != ():
	57	mode = os.stat(full_phys_dir).st_mode
	58	os.chmod(full_phys_dir, mode \| guarantee_mode_bits[0])
	59
43	60	session.resources.create(base_name,'unixfilesystem',session.host,full_phys_dir)
44	61	return full_phys_dir
45	62

48	65
49	66
50	67	from irods.test.helpers import (create_simple_resc)
	68
51	69
52	70	def setUp(self):
53	71	# Create test collection

1516	1534	self.sess.resources.remove( resc_name )
1517	1535	self.assertIs( default_XML_parser(), current_XML_parser() )
1518	1536
	1537
	1538	def test_data_open_on_leaf_is_disallowed__243(self):
	1539	from irods.exception import DIRECT_CHILD_ACCESS
	1540	root = unique_name(my_function_name(),datetime.now(),'root')
	1541	home = helpers.home_collection(self.sess)
	1542	with self.create_resc_hierarchy(root) as (_ , Leaf):
	1543	with self.assertRaises(DIRECT_CHILD_ACCESS):
	1544	self.sess.data_objects.open('{home}/disallowed_243'.format(locals()), 'w', {kw.RESC_NAME_KW:Leaf})
	1545
	1546	def test_data_open_on_named_resource__243(self):
	1547	s = self.sess
	1548	data = s.data_objects
	1549	home = helpers.home_collection(s)
	1550	data_name = unique_name(my_function_name(),datetime.now(),'data')
	1551	resc_name = unique_name(my_function_name(),datetime.now(),'resc')
	1552	with self.create_simple_resc(resc_name) as resc:
	1553	data_path = '{home}/{data_name}'.format(**locals())
	1554	try:
	1555	with data.open(data_path,'w',**{kw.RESC_NAME_KW:resc}) as f:
	1556	f.write(b'abc')
	1557	d = data.get(data_path)
	1558	self.assertEqual(len(d.replicas),1)
	1559	self.assertEqual(d.replicas[0].resource_name, resc)
	1560	finally:
	1561	if data.exists(data_path):
	1562	data.unlink(data_path, force = True)
	1563
	1564
	1565	class _temporary_resource_adopter:
	1566	"""When used as part of a context block, temporarily adopts the named
	1567	child resources under the named parent resource.
	1568	"""
	1569	def __init__(self, sess, parent, child_list=()):
	1570	self.parent = parent
	1571	self.child_list = child_list
	1572	self.sess = sess
	1573	def __enter__(self):
	1574	for child in self.child_list:
	1575	self.sess.resources.add_child(self.parent, child)
	1576	def __exit__(self, *_):
	1577	p_obj = self.sess.resources.get(self.parent)
	1578	for child in set(self.child_list) & set(r.name for r in p_obj.children):
	1579	self.sess.resources.remove_child(self.parent, child)
	1580
	1581
	1582	def test_access_through_resc_hierarchy__243(self):
	1583	s = self.sess
	1584	data_path = '{coll}/{data}'.format(coll = helpers.home_collection(s),
	1585	data = unique_name(my_function_name(),datetime.now()))
	1586	try:
	1587	s.resources.create('parent','deferred')
	1588	with self.create_simple_resc('resc0_243') as r0, \
	1589	self.create_simple_resc('resc1_243') as r1, \
	1590	self._temporary_resource_adopter(s, parent = 'parent', child_list = (r0, r1)):
	1591
	1592	hiers = ['parent;{0}'.format(r) for r in (r0,r1)]
	1593
	1594	# Write two different replicas. Although the writing of the second will cause the first to become
	1595	# stale, each replica can be independently read by specifying the resource hierarchy.
	1596	for hier in hiers:
	1597	opts = {kw.RESC_HIER_STR_KW: hier}
	1598	with s.data_objects.open(data_path, 'a', **opts) as obj_io:
	1599	obj_io.seek(0)
	1600	obj_io.write(hier.encode('utf-8')) # Write different content to each replica
	1601
	1602	# Assert that we are able to read both replicas' content faithfully using the hierarchy string.
	1603	for hier in hiers:
	1604	opts = {kw.RESC_HIER_STR_KW: hier}
	1605	with s.data_objects.open(data_path, 'r', **opts) as obj_io:
	1606	self.assertEqual(obj_io.read(),hier.encode('utf-8')) # Does unique replica have unique content?
	1607
	1608	s.data_objects.unlink(data_path, force = True)
	1609	finally:
	1610	s.resources.remove('parent')
	1611
1519	1612	def test_register_with_xml_special_chars(self):
1520	1613	test_dir = helpers.irods_shared_tmp_dir()
1521	1614	loc_server = self.sess.host in ('localhost', socket.gethostname())

1551	1644	self.sess.resources.get(resc_name).remove()
1552	1645
1553	1646
	1647	def test_unregister_can_target_replica__issue_435(self):
	1648	test_dir = helpers.irods_shared_tmp_dir()
	1649	loc_server = self.sess.host in ('localhost', socket.gethostname())
	1650	if not(test_dir) and not (loc_server):
	1651	self.skipTest('Requires access to server-side file(s)')
	1652	dt=datetime.now()
	1653	uniq1 = unique_name (my_function_name(), 1, dt)
	1654	uniq2 = unique_name (my_function_name(), 2, dt)
	1655	dir1 = dir2 = ''
	1656	data_objects = []
	1657	try:
	1658	VAULT_MODE = (loc_server, 0o777\|stat.S_ISGID)
	1659	dir1 = make_ufs_resc_in_tmpdir(self.sess, uniq1, allow_local = loc_server, client_vault_mode = VAULT_MODE)
	1660	dir2 = make_ufs_resc_in_tmpdir(self.sess, uniq2, allow_local = loc_server, client_vault_mode = VAULT_MODE)
	1661
	1662	def replica_number_from_resource_name(data_path, resc):
	1663	return [r.number for r in self.sess.data_objects.get(data_path).replicas if r.resource_name == resc][0]
	1664
	1665	# Use two different ways to specify unregister target:
	1666	for keyword in (kw.RESC_NAME_KW, kw.REPL_NUM_KW):
	1667	dt=datetime.now()
	1668	data_path = '{self.coll_path}/435_test_{dt:%s.%f}'.format(**locals())
	1669	data = self.sess.data_objects.create(data_path, resource = uniq1)
	1670	data_objects.append(data)
	1671
	1672	# Ensure that two replicas exist.
	1673	data.replicate(**{kw.DEST_RESC_NAME_KW:uniq2})
	1674	data = self.sess.data_objects.get(data_path)
	1675	self.assertEqual(2,len(data.replicas))
	1676
	1677	physical_paths = [r.path for r in data.replicas]
	1678
	1679	# Assert that unregistering the specific replica decreases the number of replicas by 1.
	1680	data.unregister(**{keyword:(replica_number_from_resource_name(data_path, uniq2) if keyword == kw.REPL_NUM_KW else uniq2),
	1681	kw.COPIES_KW:1})
	1682	self.assertEqual(1, len(self.sess.data_objects.get(data_path).replicas))
	1683
	1684	# Assert replica files still both on disk.
	1685	for phys in physical_paths:
	1686	os.stat(phys)
	1687	finally:
	1688	# Clean up.
	1689	for d in data_objects: d.unlink(force = True)
	1690	if dir1: self.sess.resources.get(uniq1).remove()
	1691	if dir2: self.sess.resources.get(uniq2).remove()
	1692
	1693	def test_set_and_access_data_comments__issue_450(self):
	1694	comment = unique_name(my_function_name(), datetime.now()) + " issue 450"
	1695	ses = self.sess
	1696	with self.create_simple_resc() as newResc:
	1697	try:
	1698	d = ses.data_objects.create('/{0.zone}/home/{0.username}/data_object_for_issue_450_test'.format(ses))
	1699	d.replicate(**{kw.DEST_RESC_NAME_KW:newResc})
	1700	ses.data_objects.modDataObjMeta({'objPath':d.path, 'rescHier':ses.resources.get(newResc).hierarchy_string},
	1701	{'dataComments':comment})
	1702	d = ses.data_objects.get(d.path)
	1703	repl = [r for r in d.replicas if r.resource_name == newResc][0]
	1704	self.assertEqual(repl.comments, comment)
	1705	finally:
	1706	d.unlink(force = True)
	1707
1554	1708	if __name__ == '__main__':
1555	1709	# let the tests find the parent irods lib
1556	1710	sys.path.insert(0, os.path.abspath('../..'))

+50

-0

irods/test/exception_test.py less more

	0	#! /usr/bin/env python
	1	from __future__ import print_function
	2	from __future__ import absolute_import
	3	from datetime import datetime
	4	import errno
	5	import os
	6	import sys
	7	import unittest
	8	import irods.test.helpers as helpers
	9	import irods.exception
	10
	11
	12	class TestException(unittest.TestCase):
	13
	14	def setUp(self):
	15	# open the session (per-test)
	16	self.sess = helpers.make_session()
	17
	18	def tearDown(self):
	19	# close the session (per-test)
	20	self.sess.cleanup()
	21
	22	def test_400(self):
	23	ses = self.sess
	24	data = ''
	25	try:
	26	seed = helpers.my_function_name() + ":" + str(datetime.now())
	27	data = helpers.home_collection(ses) + "/" + helpers.unique_name(seed,'data')
	28	exc = None
	29	with helpers.create_simple_resc(self,vault_path = '/home') as resc_name:
	30	try:
	31	# iRODS will attempt to make a data object where it doesn't have permission
	32	ses.data_objects.create(data,resource = resc_name)
	33	except Exception as e:
	34	exc = e
	35	self.assertIsNotNone(exc, 'Creating data object in unwriteable vault did not raise an error as it should.')
	36	excep_repr = repr(exc)
	37	errno_object = irods.exception.Errno(errno.EACCES)
	38	errno_repr = repr(errno_object)
	39	self.assertRegexpMatches(errno_repr,r'\bErrno\b')
	40	self.assertRegexpMatches(errno_repr,'''['"]{msg}['"]'''.format(msg = os.strerror(errno.EACCES)))
	41	self.assertIn( errno_repr, excep_repr)
	42	finally:
	43	if ses.data_objects.exists(data):
	44	ses.data_objects.unlink(data,force = True)
	45
	46	if __name__ == '__main__':
	47	# let the tests find the parent irods lib
	48	sys.path.insert(0, os.path.abspath('../..'))
	49	unittest.main()

+32

-13

irods/test/helpers.py less more

0	0	from __future__ import absolute_import
	1	from __future__ import print_function
1	2	import os
2	3	import io
3	4	import tempfile

12	13	import random
13	14	import datetime
14	15	import json
15		from pwd import getpwnam
	16	import sys
16	17	from irods.session import iRODSSession
17		from irods.message import iRODSMessage
	18	from irods.message import (iRODSMessage, IRODS_VERSION)
18	19	from irods.password_obfuscation import encode
19	20	from six.moves import range
	21
	22
	23	class StopTestsException(Exception):
	24
	25	def __init__(self,args,*kwargs):
	26	super(StopTestsException,self).__init__(args,*kwargs)
	27	if 'unittest' in sys.modules.keys():
	28	print("Aborting tests [ Got : %r ]" % self, file = sys.stderr)
	29	os.abort()
	30
	31
	32	class iRODS_Server_Too_Recent(StopTestsException):
	33	pass
20	34
21	35
22	36	def my_function_name():

87	101	return (config, auth)
88	102
89	103
90		def make_session(**kwargs):
	104	# Create a connection for test, based on ~/.irods environment by default.
	105
	106	def make_session(test_server_version = True, **kwargs):
91	107	try:
92	108	env_file = kwargs.pop('irods_env_file')
93	109	except KeyError:

95	111	env_file = os.environ['IRODS_ENVIRONMENT_FILE']
96	112	except KeyError:
97	113	env_file = os.path.expanduser('~/.irods/irods_environment.json')
98
99		try:
100		os.environ['IRODS_CI_TEST_RUN']
101		uid = getpwnam('irods').pw_uid
102		except KeyError:
103		uid = None
104
105		return iRODSSession( irods_authentication_uid = uid, irods_env_file = env_file, **kwargs )
	114	session = iRODSSession( irods_env_file = env_file, **kwargs )
	115
	116	if test_server_version:
	117	connected_version = session.server_version[:3]
	118	advertised_version = IRODS_VERSION[:3]
	119	if connected_version > advertised_version:
	120	msg = ("Connected server is {connected_version}, "
	121	"but this python-irodsclient advertises compatibility up to {advertised_version}.").format(**locals())
	122	raise iRODS_Server_Too_Recent(msg)
	123
	124	return session
106	125
107	126
108	127	def home_collection(session):

192	211	f.write(os.urandom(file_size))
193	212
194	213	@contextlib.contextmanager
195		def create_simple_resc (self, rescName = None):
	214	def create_simple_resc (self, rescName = None, vault_path = ''):
196	215	if not rescName:
197	216	rescName = 'simple_resc_' + unique_name (my_function_name() + '_simple_resc', datetime.datetime.now())
198	217	created = False

200	219	self.sess.resources.create(rescName,
201	220	'unixfilesystem',
202	221	host = self.sess.host,
203		path = '/tmp/' + rescName)
	222	path = vault_path or '/tmp/' + rescName)
204	223	created = True
205	224	yield rescName
206	225	finally:

+0

-54

~~irods/test/irods_consortium_continuous_integration_build_hook.py~~ less more

0		from __future__ import print_function
1
2		import optparse
3		import os
4		import shutil
5
6		import irods_python_ci_utilities
7
8
9		def main(output_root_directory):
10		run_tests()
11		if output_root_directory:
12		gather_xml_reports(output_root_directory)
13
14		def run_tests():
15		# prerequisites
16		install_testing_dependencies()
17		irods_python_ci_utilities.subprocess_get_output(['sudo', 'cp', '-r', '/var/lib/irods/.irods', '/home/irodsbuild/'], check_rc=True)
18		irods_python_ci_utilities.subprocess_get_output(['sudo', 'chown', '-R', 'irodsbuild', '/home/irodsbuild/.irods'], check_rc=True)
19		irods_python_ci_utilities.subprocess_get_output(['sudo', 'chmod', '-R', '777', '/etc/irods'], check_rc=True)
20
21		# test run
22		test_env = os.environ.copy()
23		test_env['IRODS_CI_TEST_RUN'] = '1'
24		irods_python_ci_utilities.subprocess_get_output(['python-coverage', 'run', 'irods/test/runner.py'], env=test_env)
25
26		# reports
27		irods_python_ci_utilities.subprocess_get_output(['python-coverage', 'report'], check_rc=True)
28		irods_python_ci_utilities.subprocess_get_output(['python-coverage', 'xml'], check_rc=True)
29
30		def install_testing_dependencies():
31		dispatch_map = {
32		'Ubuntu': install_testing_dependencies_apt,
33		}
34		try:
35		return dispatch_map[irods_python_ci_utilities.get_distribution()]()
36		except KeyError:
37		irods_python_ci_utilities.raise_not_implemented_for_distribution()
38
39		def install_testing_dependencies_apt():
40		irods_python_ci_utilities.install_os_packages(['git', 'python-prettytable', 'python-coverage', 'python-dev', 'libkrb5-dev'])
41		irods_python_ci_utilities.subprocess_get_output(['sudo', '-H', 'pip', 'install', 'gssapi'], check_rc=True)
42
43		def gather_xml_reports(output_root_directory):
44		irods_python_ci_utilities.mkdir_p(output_root_directory)
45		shutil.copy('coverage.xml', output_root_directory)
46		shutil.copytree('/tmp/python-irodsclient/test-reports', os.path.join(output_root_directory, 'test-reports'))
47
48		if __name__ == '__main__':
49		parser = optparse.OptionParser()
50		parser.add_option('--output_root_directory')
51		options, _ = parser.parse_args()
52
53		main(options.output_root_directory)

+158

-0

irods/test/json_config less more

	0	#!/usr/bin/env python3
	1
	2	# A BATS test helper.
	3	# - Add, delete, or change top level variables within JSON-format iRODS environment configuration files.
	4	# - PRESERVE, RESTORE, and PRESERVE_check operations to assert checkpoints in test setup/teardown phases.
	5	# - Allow variable abbreviation by means of omitting prefix. (Delineated by '*' in a VAR declaration.)
	6
	7	import argparse
	8	import ast
	9	from collections import OrderedDict
	10	import errno
	11	import json
	12	import os
	13	from os.path import (basename, abspath)
	14	import shelve
	15	import sys
	16
	17	tempstore = {}
	18	fds = {}
	19
	20	parser=argparse.ArgumentParser()
	21
	22	parser.add_argument('-c','--clear-store',action='store_true', help='clear store')
	23	parser.add_argument('-s','--store',action='store_true', help='print value of store')
	24	parser.add_argument('-i','--input',action='append', metavar='config_filename',
	25	help='names of files containing JSON configuration')
	26	parser.add_argument('actions', metavar='action', type=str, nargs='*',
	27	help='actions to be applied to the config files. ex:\n'
	28	'VAR="prefix*varname"' '\n'
	29	'varname=pyvalue' '\n'
	30	'RESTORE\|PRESERVE[_check]'
	31	)
	32
	33	opt = parser.parse_args()
	34
	35	store = shelve.open(os.path.expanduser('~/.store.{}'.format(basename(sys.argv[0]))))
	36
	37	class JSONUnrepresentable(Exception): pass
	38
	39	# represent OrderedDicts as normal dicts when printing structs loaded from JSON files
	40
	41	def to_dict(s):
	42	try:
	43	s.get("") # throw AttributeError for dict-incompatible types
	44	s = dict(s)
	45	except AttributeError:
	46	pass
	47	if isinstance(s,list):
	48	return [to_dict(item) for item in s]
	49	elif isinstance(s,dict):
	50	return {key:to_dict(value) for key,value in s.items()}
	51	elif isinstance(s,(int,str,float)):
	52	return s
	53	raise JSONUnrepresentable
	54
	55	if opt.store:
	56	import pprint
	57	pprint.pprint(to_dict(store))
	58	exit()
	59
	60	if opt.clear_store:
	61	store.clear()
	62
	63	def PRESERVE_check(): return PRESERVE('check')
	64	def PRESERVE_force(): return PRESERVE('force')
	65	def PRESERVE(modifier = 'no-replace'):
	66	if opt.input:
	67	for fname in opt.input:
	68	key = '\0i:'+ abspath(fname)
	69	existing = store.get(key,'')
	70	if existing and modifier == 'check':
	71	truth = (existing == tempstore[fname])
	72	assert truth, 'PRESERVE mismatch in {} - maybe due to missed or failed RESTORE'.format(fname)
	73	if modifier == 'force' or not existing:
	74	store[key] = tempstore[fname]
	75
	76	def RESTORE():
	77	input_ = [ (k[3:],j) for k,j in store.items() if k.startswith('\0i:') ]
	78	filt = None if not opt.input else lambda x:x[0] in opt.input
	79	input_ = filter(filt,input_)
	80	for f,j in input_:
	81	open(f,'w').write(json.dumps(j,indent=4))
	82	global do_savefiles
	83	do_savefiles=False
	84
	85	def VAR(v,do_store=False):
	86	if do_store:
	87	pfx,rest = v.split('*',1)
	88	store['\0v:'+rest]=pfx
	89	else:
	90	return store.get('\0v:'+v,'')+v
	91
	92	def savefiles():
	93	for fname,fd in fds.items():
	94	fd = fds[fname]
	95	fd.seek(0)
	96	fd.write(json.dumps(tempstore[fname],indent=4))
	97	fd.truncate()
	98	fd.close()
	99
	100	def loadfile(fname):
	101	s= ''
	102	try:
	103	f = open(fname,'r+')
	104	except (IOError,OSError) as exc:
	105	if exc.errno == errno.ENOENT:
	106	f = open(fname,'w+')
	107	else:
	108	raise
	109	s = f.read()
	110	if not s:
	111	j = {}
	112	else:
	113	j = json.loads(s,object_hook = OrderedDict)
	114	tempstore[fname] = j
	115	fds[fname] = f
	116
	117	funcs = {x:globals()[x] for x in ('PRESERVE',
	118	'RESTORE',
	119	'PRESERVE_check',
	120	'PRESERVE_force' )}
	121
	122	def isVARitem(item): return item.startswith('VAR=')
	123	def notVARitem(item): return not(isVARitem(item))
	124
	125	do_savefiles=True
	126
	127	def main():
	128
	129	if opt.input:
	130	for s in opt.input:
	131	loadfile(s)
	132
	133	for c in filter(isVARitem,opt.actions):
	134	name,value = c.split('=',1)
	135	VAR(value,True)
	136
	137	# special case - RESTORE all items in store regardless of argv input
	138	if 'RESTORE' in opt.actions and not fds:
	139	RESTORE()
	140
	141	for fname,f in fds.items():
	142	j = tempstore[fname]
	143	for c in filter(notVARitem,opt.actions):
	144	name,value = c.split('=',1) if '=' in c else (c,'')
	145	if name in funcs:
	146	funcs[name]() # RESTORE, PRESERVE[_*] for the list of fds active
	147	continue
	148	if not value:
	149	j.pop(VAR(name),None)
	150	else:
	151	j[VAR(name)] = ast.literal_eval(value)
	152
	153	if do_savefiles:
	154	savefiles()
	155
	156	if __name__ == '__main__':
	157	main()

+1

-1

irods/test/login_auth_test.py less more

383	383
384	384	admin.collections.create(self.home)
385	385	acl = iRODSAccess('own', self.home, user.name)
386		admin.permissions.set(acl)
	386	admin.acls.set(acl, admin = True)
387	387
388	388	self.env_file = os.path.expanduser('~/.irods.anon/irods_environment.json')
389	389	self.env_dir = ( os.path.dirname(self.env_file))

+91

-5

irods/test/meta_test.py less more

7	7	import unittest
8	8	from irods.meta import (iRODSMeta, AVUOperation, BadAVUOperationValue, BadAVUOperationKeyword)
9	9	from irods.manager.metadata_manager import InvalidAtomicAVURequest
10		from irods.models import (DataObject, Collection, Resource)
	10	from irods.models import (DataObject, Collection, Resource, CollectionMeta)
11	11	import irods.test.helpers as helpers
12	12	import irods.keywords as kw
13	13	from irods.session import iRODSSession
14	14	from six.moves import range
15		from six import PY3
16
	15	from six import PY2, PY3
	16	from irods.message import Bad_AVU_Field
	17	from irods.column import Like, NotLike
	18
	19
	20	def normalize_to_bytes(string, unicode_encoding='utf8'):
	21	# Python2 and 3 enumerate bytestrings differently.
	22	ord_ = (lambda _:_) if any(x for x in string[:1] if type(x) is int) else ord
	23	if not string or ord_(max(x for x in string)) > 255:
	24	return string.encode(unicode_encoding)
	25	# str->bytes type conversion using element-wise copy, aka a trivial encode.
	26	array = bytearray(ord_(x) for x in string)
	27	return bytes(array)
	28
	29	def resolves_to_identical_bytestrings(avu1, avu2, key = normalize_to_bytes):
	30	avu1 = tuple(avu1)
	31	avu2 = tuple(avu2)
	32	if len(avu1) != len(avu2):
	33	return False
	34	for field1,field2 in zip(avu1,avu2):
	35	if key(field1) != key(field2):
	36	return False
	37	return True
17	38
18	39	class TestMeta(unittest.TestCase):
19	40	'''Suite of tests on metadata operations

22	43	attr0, value0, unit0 = 'attr0', 'value0', 'unit0'
23	44	attr1, value1, unit1 = 'attr1', 'value1', 'unit1'
24	45
	46	def test_stringtypes_in_general_query__issue_442(self):
	47	metadata = []
	48	value = u'a\u1000b'
	49	value_encoded = value.encode('utf8')
	50	contains_value = {type(value_encoded):b'%%'+value_encoded+b'%%',
	51	type(value):'%%'+value+'%%'}
	52	for v in (value, value_encoded):
	53
	54	# Establish invariant of exactly 2 AVUs attached to object.
	55	self.coll.metadata.remove_all()
	56	self.coll.metadata['a']=iRODSMeta('a',value)
	57	self.coll.metadata['b']=iRODSMeta('b','<arbitrary>')
	58	q = self.sess.query(CollectionMeta).filter(Collection.name == self.coll_path)
	59	self.assertEqual(len(list(q)),2)
	60
	61	# Test query with operators Like and NotLike
	62	q = self.sess.query(CollectionMeta).filter(Collection.name == self.coll_path,
	63	NotLike(CollectionMeta.value,contains_value[type(v)]))
	64	self.assertEqual(len(list(q)),1)
	65	q = self.sess.query(CollectionMeta).filter(Collection.name == self.coll_path,
	66	Like(CollectionMeta.value,contains_value[type(v)]))
	67	self.assertEqual(len(list(q)),1)
	68
	69	# Test query with operators == and !=
	70	q = self.sess.query(CollectionMeta).filter(Collection.name == self.coll_path, CollectionMeta.value == v)
	71	self.assertEqual(len(list(q)),1)
	72	q = self.sess.query(CollectionMeta).filter(Collection.name == self.coll_path, CollectionMeta.value != v)
	73	self.assertEqual(len(list(q)),1)
	74	metadata.append(self.coll.metadata['a'])
	75
	76	# Test that application of unicode and bytestring metadata were equivalent
	77	self.assertEqual(metadata[0], metadata[1])
	78
	79	@unittest.skipIf(PY3, 'Unicode strings are normal on Python3')
	80	def test_unicode_AVUs_in_Python2__issue_442(self):
	81	data_object = self.sess.data_objects.get(self.obj_path)
	82	meta_set = iRODSMeta(u'\u1000', u'value', u'units')
	83	meta_add = iRODSMeta(*tuple(meta_set))
	84	meta_add.name += u"-add"
	85	data_object.metadata.set(meta_set)
	86	data_object.metadata.add(meta_add)
	87	for index, meta in [(m.name, m) for m in (meta_add, meta_set)]:
	88	fetched = data_object.metadata[index]
	89	self.assertTrue(resolves_to_identical_bytestrings(fetched, meta), 'fetched unexpected meta for %r' % index)
	90
	91	@unittest.skipIf(PY2, 'Byte strings are normal on Python2')
	92	def test_bytestring_AVUs_in_Python3__issue_442(self):
	93	data_object = self.sess.data_objects.get(self.obj_path)
	94	meta_set = iRODSMeta(u'\u1000'.encode('utf8'),b'value',b'units')
	95	meta_add = iRODSMeta(*tuple(meta_set))
	96	meta_add.name += b"-add"
	97	data_object.metadata.set(meta_set)
	98	data_object.metadata.add(meta_add)
	99	for index, meta in [(m.name, m) for m in (meta_add, meta_set)]:
	100	fetched = data_object.metadata[index]
	101	self.assertTrue(resolves_to_identical_bytestrings(fetched, meta), 'fetched unexpected meta for %r' % index)
	102
25	103	def setUp(self):
26	104	self.sess = helpers.make_session()
27	105	# test data

44	122
45	123	def test_atomic_metadata_operations_244(self):
46	124	user = self.sess.users.get("rods")
47		group = self.sess.user_groups.get("public")
	125	group = self.sess.groups.get("public")
48	126	m = ( "attr_244","value","units")
49	127
50	128	with self.assertRaises(BadAVUOperationValue):

107	185	def test_atomic_metadata_operations_255(self):
108	186	my_resc = self.sess.resources.create('dummyResc','passthru')
109	187	avus = [iRODSMeta('a','b','c'), iRODSMeta('d','e','f')]
110		objects = [ self.sess.users.get("rods"), self.sess.user_groups.get("public"), my_resc,
	188	objects = [ self.sess.users.get("rods"), self.sess.groups.get("public"), my_resc,
111	189	self.sess.collections.get(self.coll_path), self.sess.data_objects.get(self.obj_path) ]
112	190	try:
113	191	for obj in objects:

441	519	if d: d.unlink(force = True)
442	520	helpers.remove_unused_metadata(session)
443	521
	522	def test_nonstring_as_AVU_value_raises_an_error__issue_434(self):
	523	with self.assertRaisesRegexp(Bad_AVU_Field,'incorrect type'):
	524	self.coll.metadata.set("an_attribute",0)
	525
	526	def test_empty_string_as_AVU_value_raises_an_error__issue_434(self):
	527	with self.assertRaisesRegexp(Bad_AVU_Field,'zero-length'):
	528	self.coll.metadata.set("an_attribute","")
	529
444	530	if __name__ == '__main__':
445	531	# let the tests find the parent irods lib
446	532	sys.path.insert(0, os.path.abspath('../..'))

+13

-1

irods/test/query_test.py less more

18	18	from tempfile import NamedTemporaryFile
19	19	from irods.exception import MultipleResultsFound, CAT_UNKNOWN_SPECIFIC_QUERY, CAT_INVALID_ARGUMENT
20	20	from irods.query import SpecificQuery
21		from irods.column import Like, Between, In
	21	from irods.column import Like, NotLike, Between, In
22	22	from irods.meta import iRODSMeta
23	23	from irods.rule import Rule
24	24	from irods import MAX_SQL_ROWS

539	539	if results:
540	540	Rule(self.sess).remove_by_id( results[0][RuleExec.id] )
541	541
	542	def test_utf8_equality_in_query__issue_442(self):
	543	name = u'prefix-\u1000-suffix'
	544	self.sess.data_objects.create(u'{}/{}'.format(self.coll_path, name))
	545	query = self.sess.query(DataObject).filter( DataObject.name == name.encode('utf8'),
	546	DataObject.collection_id == self.coll.id)
	547	self.assertEqual(1, len(list(query)))
	548
	549	def test_not_like_operator__issue_443(self):
	550	name_search_pattern = '%_issue_443.non_matching'
	551	query = self.sess.query(DataObject).filter( NotLike(DataObject.name, name_search_pattern),
	552	DataObject.collection_id == self.coll.id)
	553	self.assertEqual(1, len(list(query)))
542	554
543	555	class TestSpecificQuery(unittest.TestCase):
544	556

+62

-0

irods/test/resource_test.py less more

	0	#! /usr/bin/env python
	1	from __future__ import absolute_import
	2	import os
	3	import six
	4	import sys
	5	import unittest
	6
	7	from irods.test import helpers
	8
	9	class TestResource(unittest.TestCase):
	10
	11	from helpers import create_simple_resc_hierarchy
	12
	13	def setUp(self):
	14	self.sess = helpers.make_session()
	15
	16	def test_resource_properties_for_parent_and_hierarchy_at_3_levels__392(self):
	17	ses = self.sess
	18	root = "deferred_392"
	19	pt = "pt_392"
	20	leaf = "leaf_392"
	21	root_resc = ses.resources.create(root,"deferred")
	22	try:
	23	# Create two (passthru + storage) hierarchies below the root: ie. pt0;leaf0 and pt1;leaf1
	24	with self.create_simple_resc_hierarchy(pt + "_0", leaf + "_0"), \
	25	self.create_simple_resc_hierarchy(pt + "_1", leaf + "_1"):
	26	try:
	27	# Adopt both passthru's as children under the main root (deferred) node.
	28	ses.resources.add_child(root, pt + "_0")
	29	ses.resources.add_child(root, pt + "_1")
	30	# Now we have two different 3-deep hierarchies (root;pt0;leaf0 and root;pt1;leaf) sharing the same root node.
	31	# Descend each and make sure the relationships hold
	32	for mid in root_resc.children:
	33	hierarchy = [root_resc, mid, mid.children[0]]
	34	parent_resc = None
	35	hier_str = root
	36	# Assert that the hierarchy and parent properties hold at each level, in both tree branches.
	37	for n,resc in enumerate(hierarchy):
	38	if n > 0:
	39	hier_str += ";{}".format(resc.name)
	40	self.assertEqual(resc.parent_id, (None if n == 0 else parent_resc.id))
	41	self.assertEqual(resc.parent_name, (None if n == 0 else parent_resc.name))
	42	self.assertEqual(resc.hierarchy_string, hier_str)
	43	self.assertIs(type(resc.hierarchy_string), str) # type of hierarchy field is string.
	44	if resc.parent is None:
	45	self.assertIs(resc.parent_id, None)
	46	self.assertIs(resc.parent_name, None)
	47	else:
	48	self.assertIn(type(resc.parent_id), six.integer_types) # type of a non-null id field is integer.
	49	self.assertIs(type(resc.parent_name), str) # type of a non-null name field is string.
	50	parent_resc = resc
	51	finally:
	52	ses.resources.remove_child(root, pt + "_0")
	53	ses.resources.remove_child(root, pt + "_1")
	54	finally:
	55	ses.resources.remove(root)
	56
	57
	58	if __name__ == '__main__':
	59	# let the tests find the parent irods lib
	60	sys.path.insert(0, os.path.abspath('../..'))
	61	unittest.main()

+38

-15

irods/test/setupssl.py less more

8	8	from subprocess import (Popen, PIPE)
9	9
10	10	IRODS_SSL_DIR = '/etc/irods/ssl'
	11	SERVER_CERT_HOSTNAME = None
	12	ext=''
	13	keep_old = False
11	14
12		def create_ssl_dir():
	15	def create_server_cert(process_output = sys.stdout, irods_key_path = 'irods.key', hostname = SERVER_CERT_HOSTNAME):
	16	p = Popen("openssl req -new -x509 -key '{irods_key_path}' -out irods.crt{ext} -days 365 <<EOF{_sep_}"
	17	"US{_sep_}North Carolina{_sep_}Chapel Hill{_sep_}UNC{_sep_}RENCI{_sep_}"
	18	"{host}{_sep_}anon@mail.com{_sep_}EOF\n""".format(
	19	ext = ext,
	20	host = (hostname if hostname else socket.gethostname()),
	21	_sep_ = "\n",
	22	**locals()),
	23	shell = True, stdout = process_output, stderr = process_output)
	24	p.wait()
	25	return p.returncode
	26
	27	def create_ssl_dir(irods_key_path = 'irods.key'):
13	28	save_cwd = os.getcwd()
14	29	silent_run = { 'shell': True, 'stderr' : PIPE, 'stdout' : PIPE }
15	30	try:
16	31	if not (os.path.exists(IRODS_SSL_DIR)):
17	32	os.mkdir(IRODS_SSL_DIR)
18	33	os.chdir(IRODS_SSL_DIR)
19		Popen("openssl genrsa -out irods.key 2048",**silent_run).communicate()
	34	if not keep_old:
	35	Popen("openssl genrsa -out '{irods_key_path}' 2048 && chmod 600 '{irods_key_path}'".format(**locals()),
	36	**silent_run).communicate()
20	37	with open("/dev/null","wb") as dev_null:
21		p = Popen("openssl req -new -x509 -key irods.key -out irods.crt -days 365 <<EOF{_sep_}"
22		"US{_sep_}North Carolina{_sep_}Chapel Hill{_sep_}UNC{_sep_}RENCI{_sep_}"
23		"{host}{_sep_}anon@mail.com{_sep_}EOF\n""".format(
24		host = socket.gethostname(), _sep_="\n"),shell=True, stdout=dev_null, stderr=dev_null)
25		p.wait()
26		if 0 == p.returncode:
27		Popen('openssl dhparam -2 -out dhparams.pem',**silent_run).communicate()
	38	if 0 == create_server_cert(process_output = dev_null, irods_key_path = irods_key_path):
	39	if not keep_old:
	40	Popen('openssl dhparam -2 -out dhparams.pem',**silent_run).communicate()
28	41	return os.listdir(".")
29	42	finally:
30	43	os.chdir(save_cwd)
31	44
32		def test(opts,args=()):
33		if args: print ('warning: non-option args are ignored',file=sys.stderr)
34		affirm = 'n' if os.path.exists(IRODS_SSL_DIR) else 'y'
35		if not [v for k,v in opts if k == '-f'] and affirm == 'n' and posix.isatty(sys.stdin.fileno()):
	45	def test(options, args=()):
	46	if args:
	47	print ('warning: non-option args are ignored',file=sys.stderr)
	48	force = ('-f' in options)
	49	affirm = 'n' if (os.path.exists(IRODS_SSL_DIR) and not force) else 'y'
	50	if affirm == 'n' and posix.isatty(sys.stdin.fileno()):
36	51	try:
37	52	input_ = raw_input
38	53	except NameError:
39	54	input_ = input
40	55	affirm = input_("This will overwrite directory '{}'. Proceed(Y/N)? ".format(IRODS_SSL_DIR))
41	56	if affirm[:1].lower() == 'y':
42		shutil.rmtree(IRODS_SSL_DIR,ignore_errors=True)
	57	if not keep_old:
	58	shutil.rmtree(IRODS_SSL_DIR,ignore_errors=True)
43	59	print("Generating new '{}'. This may take a while.".format(IRODS_SSL_DIR), file=sys.stderr)
44	60	ssl_dir_files = create_ssl_dir()
45	61	print('ssl_dir_files=', ssl_dir_files)
46	62
47	63	if __name__ == '__main__':
48	64	import getopt
49		test(*getopt.getopt(sys.argv[1:],'f')) # f = force
	65	opt, arg_list = getopt.getopt(sys.argv[1:],'x:fh:k')
	66	opt_lookup = dict(opt)
	67	ext = opt_lookup.get('-x','')
	68	if ext:
	69	ext = '.' + ext.lstrip('.')
	70	keep_old = opt_lookup.get('-k') is not None
	71	SERVER_CERT_HOSTNAME = opt_lookup.get('-h')
	72	test(opt_lookup, arg_list)

+13

-0

irods/test/ssl_test_client.py less more

	0	from irods.test.helpers import (make_session, home_collection)
	1	import ssl
	2
	3	# Exercise an API to instantiate a connection.
	4	session = make_session()
	5	home = home_collection(session)
	6	session.collections.get(home)
	7
	8	# Assert that the resulting connection is SSL-enabled.
	9	connections = session.pool.active \| session.pool.idle
	10	is_SSL = len(connections) > 0 and \
	11	all(isinstance(conn.socket, ssl.SSLSocket) for conn in connections)
	12	exit(0 if is_SSL else 1)

+172

-0

irods/test/test_ssl_context.bats less more

	0	#!/usr/bin/env bats
	1
	2	# Run with
	3	# * $RUN containing a unique string
	4	# * $REPO pointing to this repository
	5	# For example, in Bash:
	6	# $ REPO=~/relative/path/python-irodsclient RUN=$$:`date +%s` bats test_ssl_context.bats
	7	# (This allows us to perform the one-time initialization before test cases are run.)
	8	#
	9	# Note also:
	10	#
	11	# This series of tests should be run by a Linux user (not root, and not the iRODS service
	12	# account i.e. irods) iinit'ed as rods in a default iRODS server installation.
	13	#
	14	# That user's home directory, the python-irodsclient repository, and all intervening
	15	# path elements need to be visible to the irods user (a concern on Centos 7).
	16	#
	17	# The bats package must be installed to run this test script.
	18	#
	19	# The c_rehash binary is also needed (provided by the openssl package on Debian-like and
	20	# openssl-perl on RHEL-like operating systems.)
	21
	22	# iRODS RELATED INITIALIZATION
	23
	24	IRODS_LOCAL_ENV=~/.irods/irods_environment.json
	25	IRODS_ACCOUNT_ENV=~irods/.irods/irods_environment.json
	26
	27	edit_core_re () {
	28	if [ "$1" = ssl ]; then
	29	sudo su irods -c "sed -i.orig 's/$^\sacPreConnect.CS_NEG$$[A-Z_]*$/\1_REQUIRE/' /etc/irods/core.re"
	30	else
	31	if [ -f /etc/irods/core.re.orig ]; then
	32	sudo su irods -c "cp -rp /etc/irods/core.re.orig /etc/irods/core.re"
	33	else
	34	echo >&2 "Warning - could not restore original core.re"
	35	fi
	36	fi
	37	}
	38
	39	restart_server()
	40	{
	41	sudo su irods -c '~/irodsctl restart'
	42	}
	43
	44	if [ "$LOGFILE" = "<syslog>" ]; then
	45	log () { logger "`date`: $*"; } # Log to (r)syslog
	46	elif [ -n "$LOGFILE" ]; then
	47	log () { echo "`date`: $*" >>"$LOGFILE" ; } # Log to a file.
	48	else
	49	log () { :; } # NOP
	50	fi
	51
	52	: ${REPO:=~/python-irodsclient}
	53	REPO_SCRIPTS="$REPO/irods/test"
	54	PATH=$REPO_SCRIPTS:$PATH
	55
	56	ABBREVIATIONS=(
	57	VAR='irods_ssl_*ca_certificate_file'
	58	VAR='irods_ssl_*ca_certificate_path'
	59	VAR='irods_ssl_*verify_server'
	60	VAR='irods_ssl_*dh_params_file'
	61	VAR='irods_ssl_*certificate_key_file'
	62	VAR='irods_ssl_*certificate_chain_file'
	63	VAR='irods_*encryption_algorithm'
	64	VAR='irods_*encryption_key_size'
	65	VAR='irods_*encryption_num_hash_rounds'
	66	VAR='irods_*encryption_salt_size'
	67	VAR='irods_*client_server_policy'
	68	VAR='irods_*client_server_negotiation'
	69	)
	70
	71	touch /tmp/run
	72	if [ "`cat /tmp/run`" != "$RUN" ]; then
	73
	74	## -- Begin one-time initialization --
	75
	76	# Initialize the variable abbreviations
	77	json_config --clear-store ${ABBREVIATIONS[*]}
	78	# The next two lines were necessary under Centos 7. sudo behaved differently wrt
	79	# what is considered the home directory, so the wrong ~/.store* file was being used.
	80	sudo su irods -c "$REPO_SCRIPTS/json_config --clear-store ${ABBREVIATIONS[*]}"
	81	sudo $REPO_SCRIPTS/json_config --clear-store ${ABBREVIATIONS[*]}
	82
	83	# Set up the basic server cert, key, and DH params file.
	84	[ -e /etc/irods/ssl ] \|\| sudo su irods -c "$REPO_SCRIPTS/setupssl.py -f"
	85
	86	# Set up another cert with non-matching hostname.
	87	sudo su irods -c "$REPO_SCRIPTS/setupssl.py -kf -x.localhost -hlocalhost"
	88	sudo su irods -c "c_rehash /etc/irods/ssl"
	89
	90	# Change the iRODS svc account user's (and current user's) iRODS environment file for SSL.
	91	sudo $REPO_SCRIPTS/json_config -i $IRODS_ACCOUNT_ENV\
	92	'client_server_policy="CS_NEG_REQUIRE"'\
	93	'ca_certificate_file="/etc/irods/ssl/irods.crt"'\
	94	'certificate_key_file="/etc/irods/ssl/irods.key"'\
	95	'dh_params_file="/etc/irods/ssl/dhparams.pem"'\
	96	'certificate_chain_file="/etc/irods/ssl/irods.crt"'\
	97	'verify_server="cert"'
	98	json_config -i $IRODS_LOCAL_ENV\
	99	'client_server_negotiation="request_server_negotiation"'\
	100	'encryption_algorithm="AES-256-CBC"'\
	101	'encryption_key_size=32'\
	102	'encryption_num_hash_rounds=16'\
	103	'encryption_salt_size=8'\
	104	'client_server_policy="CS_NEG_REQUIRE"'\
	105	'verify_server="cert"'\
	106	'ca_certificate_file="/etc/irods/ssl/irods.crt"'
	107
	108	# Set the SSL-reconfigured environment files as (PRESERVE/RESTORE) checkpoints
	109	# to be managed by setup and teardown.
	110	sudo $REPO_SCRIPTS/json_config -i $IRODS_ACCOUNT_ENV -i $IRODS_LOCAL_ENV PRESERVE
	111
	112	restart_server
	113
	114	edit_core_re ssl
	115
	116	# In case of things falling down prematurely, set things back to a stable state.
	117	# This is an unconditional and one-time finalization that runs regardless, after all tests
	118	# or in case of something catastrophic such as being killed by a signal.)
	119	trap 'log "Tests Finalizing..."
	120	sudo $REPO_SCRIPTS/json_config -i $IRODS_ACCOUNT_ENV -i $IRODS_LOCAL_ENV RESTORE
	121	edit_core_re RESTORE
	122	' exit
	123	#
	124	## -- End one-time init --
	125	echo "$RUN" >/tmp/run
	126	fi
	127
	128	# TEST-RELATED SETUP & TEARDOWN
	129
	130	setup() {
	131	log "[$BATS_TEST_NAME] - setup"
	132	# Make sure we're back to the configuration checkpoint.
	133	sudo $REPO_SCRIPTS/json_config -i "$IRODS_ACCOUNT_ENV" -i "$IRODS_LOCAL_ENV" PRESERVE_check
	134	}
	135
	136	teardown() {
	137	log "[$BATS_TEST_NAME] - teardown"
	138	# Restore to the configuration checkpoint.
	139	sudo $REPO_SCRIPTS/json_config -i "$IRODS_ACCOUNT_ENV" -i "$IRODS_LOCAL_ENV" RESTORE
	140	}
	141
	142	# THE TESTS THEMSELVES
	143
	144	@test "basic_test" {
	145	json_config -i $IRODS_LOCAL_ENV 'verify_server="host"'
	146	python3 $REPO_SCRIPTS/ssl_test_client.py
	147	}
	148
	149	@test "capath_test" {
	150	json_config -i $IRODS_LOCAL_ENV 'ca_certificate_path="/etc/irods/ssl"'\
	151	'ca_certificate_file='
	152	python3 $REPO_SCRIPTS/ssl_test_client.py
	153	}
	154
	155	@test "nocerts_test" {
	156	json_config -i $IRODS_LOCAL_ENV 'ca_certificate_path='\
	157	'ca_certificate_file='\
	158	'verify_server="none"'
	159	python3 $REPO_SCRIPTS/ssl_test_client.py
	160	}
	161
	162	@test "non_matching_hostname_test" {
	163	local CERT_NOT_MATCHING_HOSTNAME=/etc/irods/ssl/irods.crt.localhost
	164	sudo $REPO_SCRIPTS/json_config -i $IRODS_LOCAL_ENV $IRODS_ACCOUNT_ENV\
	165	'verify_server="cert"'\
	166	"ca_certificate_file='$CERT_NOT_MATCHING_HOSTNAME'"
	167	sudo $REPO_SCRIPTS/json_config -i $IRODS_ACCOUNT_ENV\
	168	"certificate_chain_file='$CERT_NOT_MATCHING_HOSTNAME'"
	169	restart_server
	170	python3 $REPO_SCRIPTS/ssl_test_client.py
	171	}

+213

-23

irods/test/user_group_test.py less more

0	0	#! /usr/bin/env python
1	1	from __future__ import absolute_import
	2	import datetime
2	3	import os
3	4	import sys
4	5	import unittest
5	6	import tempfile
6	7	import shutil
7		from irods.exception import UserGroupDoesNotExist, UserDoesNotExist
	8	from irods import MAX_PASSWORD_LENGTH
	9	from irods.exception import GroupDoesNotExist, UserDoesNotExist
8	10	from irods.meta import iRODSMetaCollection, iRODSMeta
9		from irods.models import User, UserGroup, UserMeta
	11	from irods.models import User, Group, UserMeta
10	12	from irods.session import iRODSSession
11	13	import irods.exception as ex
12	14	import irods.test.helpers as helpers
	15	from irods.user import Bad_password_change_parameter
13	16	from six.moves import range
14	17
15
16		class TestUserGroup(unittest.TestCase):
	18	class TestGroup(unittest.TestCase):
17	19
18	20	def setUp(self):
19	21	self.sess = helpers.make_session()

85	87	shutil.rmtree(ENV_DIR)
86	88	ses.users.remove('alice')
87	89
88		def test_modify_password_with_incorrect_old_value__328(self):
	90	def test_modifying_password_at_various_lengths__issue_328(self):
	91	ses = self.sess
	92	try:
	93	pw_lengths = [3, # minimum password length
	94	MAX_PASSWORD_LENGTH - 8, # maximum password length
	95	26] # one greater than threshold (See irods/irods#7084)
	96
	97	# Test different combinations of new and old password lengths
	98	tuples_of_old_and_new_password = [('a'x,'b'y) for x in pw_lengths for y in pw_lengths]
	99	ses.users.create('alice', 'rodsuser')
	100
	101	for old_pw, new_pw in tuples_of_old_and_new_password:
	102	ses.users.modify('alice', 'password',old_pw)
	103	# Successful change of own password
	104	with iRODSSession(user = 'alice', password = old_pw, host = ses.host, port = ses.port, zone = ses.zone) as user_sess:
	105	user_sess.users.modify_password(old_pw, new_pw)
	106	# Test new password is usable
	107	with iRODSSession(user = 'alice', password = new_pw, host = ses.host, port = ses.port, zone = ses.zone) as user_sess:
	108	self.do_something(user_sess)
	109	finally:
	110	ses.users.remove('alice')
	111
	112	def test_password_corruption_can_be_prevented__issue_448(self):
	113	ses = self.sess
	114	try:
	115	# Threshold value of new password length, at which the server cannot
	116	# properly detect the old-password value is incorrect).
	117	# this is because its descrambled value does not contain space
	118	# for the trailing sixteen character pattern needed by the server to detect a
	119	# valid result for descramble. Of course, it goes ahead and modifies
	120	# the database anyway....so we must screen for this in the client interface.
	121	# (See irods/irods#7084)
	122
	123	threshold = 25
	124
	125	max_pw_len = MAX_PASSWORD_LENGTH - 8
	126	tuples_of_old_and_new_password = [ ('a'24,'b'threshold),
	127	('a'24,'b'(max_pw_len)),
	128	('a'24,'b'(max_pw_len + 1)) ]
	129	ses.users.create('alice', 'rodsuser')
	130	for old_pw, new_pw in tuples_of_old_and_new_password:
	131	ses.users.modify('alice', 'password', old_pw)
	132
	133	# Test that we catch the attempt to change the old password when providing incorrect old password
	134	with self.assertRaises(Bad_password_change_parameter):
	135	with iRODSSession(user = 'alice', password = old_pw, host = ses.host, port = ses.port, zone = ses.zone) as user_sess:
	136	user_sess.users.modify_password(old_pw+".", new_pw)
	137
	138	# Test that we can still change to a valid new password and that the new login is useable.
	139	new_pw = new_pw[:max_pw_len] #-> forcing to valid length
	140	with iRODSSession(user = 'alice', password = old_pw, host = ses.host, port = ses.port, zone = ses.zone) as user_sess:
	141	user_sess.users.modify_password(old_pw, new_pw)
	142	with iRODSSession(user = 'alice', password = new_pw, host = ses.host, port = ses.port, zone = ses.zone) as user_sess:
	143	self.do_something(user_sess)
	144	finally:
	145	ses.users.remove('alice')
	146
	147	def test_modify_password_with_incorrect_old_value__issue_328(self):
89	148	ses = self.sess
90	149	if ses.users.get( ses.username ).type != 'rodsadmin':
91	150	self.skipTest( 'Only a rodsadmin may run this test.')

104	163	for factory in session_factories:
105	164	with factory() as alice_ses:
106	165	alice = alice_ses.users.get(alice_ses.username)
107		with self.assertRaises( ex.CAT_PASSWORD_ENCODING_ERROR ):
	166	with self.assertRaises(Bad_password_change_parameter):
108	167	alice.modify_password(OLDPASS + ".", NEWPASS)
109	168	with iRODSSession(**d) as alice_ses:
110	169	self.do_something(alice_ses)

116	175	group_name = "test_group"
117	176
118	177	# group should not be already present
119		with self.assertRaises(UserGroupDoesNotExist):
120		self.sess.user_groups.get(group_name)
	178	with self.assertRaises(GroupDoesNotExist):
	179	self.sess.groups.get(group_name)
121	180
122	181	# create group
123		group = self.sess.user_groups.create(group_name)
	182	group = self.sess.groups.create(group_name)
124	183
125	184	# assertions
126	185	self.assertEqual(group.name, group_name)
127	186	self.assertEqual(
128		repr(group), "<iRODSUserGroup {0} {1}>".format(group.id, group_name))
	187	repr(group), "<iRODSGroup {0} {1}>".format(group.id, group_name))
129	188
130	189	# delete group
131	190	group.remove()
132	191
133	192	# group should be gone
134		with self.assertRaises(UserGroupDoesNotExist):
135		self.sess.user_groups.get(group_name)
	193	with self.assertRaises(GroupDoesNotExist):
	194	self.sess.groups.get(group_name)
136	195
137	196	def test_add_users_to_group(self):
138	197	group_name = "test_group"

140	199	base_user_name = "test_user"
141	200
142	201	# group should not be already present
143		with self.assertRaises(UserGroupDoesNotExist):
144		self.sess.user_groups.get(group_name)
	202	with self.assertRaises(GroupDoesNotExist):
	203	self.sess.groups.get(group_name)
145	204
146	205	# create test group
147		group = self.sess.user_groups.create(group_name)
	206	group = self.sess.groups.create(group_name)
148	207
149	208	# create test users names
150	209	test_user_names = []

162	221	# compare lists
163	222	self.assertSetEqual(set(member_names), set(test_user_names))
164	223
165		# exercise iRODSUserGroup.hasmember()
	224	# exercise iRODSGroup.hasmember()
166	225	for test_user_name in test_user_names:
167	226	self.assertTrue(group.hasmember(test_user_name))
168	227

176	235	group.remove()
177	236
178	237	# group should be gone
179		with self.assertRaises(UserGroupDoesNotExist):
180		self.sess.user_groups.get(group_name)
	238	with self.assertRaises(GroupDoesNotExist):
	239	self.sess.groups.get(group_name)
181	240
182	241	def test_user_dn(self):
183	242	# https://github.com/irods/irods/issues/3620

214	273	group_name = "test_group"
215	274
216	275	# group should not be already present
217		with self.assertRaises(UserGroupDoesNotExist):
218		self.sess.user_groups.get(group_name)
	276	with self.assertRaises(GroupDoesNotExist):
	277	self.sess.groups.get(group_name)
219	278
220	279	group = None
221	280
222	281	try:
223	282	# create group
224		group = self.sess.user_groups.create(group_name)
	283	group = self.sess.groups.create(group_name)
225	284
226	285	# add metadata to group
227	286	triple = ['key', 'value', 'unit']
228	287	group.metadata[triple[0]] = iRODSMeta(*triple)
229	288
230		result = self.sess.query(UserMeta, UserGroup).filter(UserGroup.name == group_name,
231		UserMeta.name == 'key').one()
	289	result = self.sess.query(UserMeta, Group).filter(Group.name == group_name,
	290	UserMeta.name == 'key').one()
232	291
233	292	self.assertTrue([result[k] for k in (UserMeta.name, UserMeta.value, UserMeta.units)] == triple)
234	293

335	394	user.remove()
336	395	helpers.remove_unused_metadata(self.sess)
337	396
	397	def create_groupadmin_user_and_session(self, groupadmin_user_name):
	398
	399	# Generate a random password.
	400	ga_password = helpers.unique_name(helpers.my_function_name(),
	401	datetime.datetime.now())[:MAX_PASSWORD_LENGTH]
	402
	403	# Create a groupadmin user with that password, and a session object for logging in as that user.
	404	groupadmin = self.sess.users.create(groupadmin_user_name, 'groupadmin')
	405	self.sess.users.modify(groupadmin_user_name, 'password', ga_password)
	406	session = iRODSSession(user = groupadmin_user_name,
	407	password = ga_password,
	408	host = self.sess.host,
	409	port = self.sess.port,
	410	zone = self.sess.zone)
	411
	412	# Return two objects: the user and the session.
	413	return (groupadmin, session)
	414
	415	def test_group_admin_can_create_and_administer_groups__issue_426(self):
	416	admin = self.sess
	417	alice = groupadmin = lab = None
	418	GROUP_ADMIN = 'groupadmin_426'
	419	try:
	420	# Create a rodsuser and a groupadmin.
	421	alice = admin.users.create('alice','rodsuser')
	422	alice.modify('password', 'apass')
	423	groupadmin, groupadmin_session = self.create_groupadmin_user_and_session(GROUP_ADMIN)
	424
	425	# As the groupadmin:
	426	# * Add two users to the group (one being the groupadmin) and assert membership.
	427	# * Remove those users from the group, and assert they are no longer members.
	428	with groupadmin_session:
	429	lab = groupadmin_session.groups.create('lab')
	430	groupadmin_session.groups.addmember('lab',GROUP_ADMIN)
	431	groupadmin_session.groups.addmember('lab','alice')
	432
	433	# Check that our members got added.
	434	# (For set objects in Python, s1 <= s2 comparison means "is s1 a subset of s2?")
	435	self.assertLessEqual(set(('alice',GROUP_ADMIN)), set(member.name for member in lab.members))
	436
	437	groupadmin_session.groups.removemember('lab','alice')
	438	groupadmin_session.groups.removemember('lab',GROUP_ADMIN)
	439
	440	# Check that our members got removed.
	441	self.assertFalse(set(('alice',GROUP_ADMIN)) & set(member.name for member in lab.members))
	442	finally:
	443	if groupadmin:
	444	groupadmin.remove()
	445	# NB: groups and users, even if created by a groupadmin, must be removed by a rodsadmin.
	446	if alice:
	447	alice.remove()
	448	if lab:
	449	admin.groups.remove(lab.name)
	450
	451	def test_group_admin_can_create_users__issue_428(self):
	452	admin = self.sess
	453	if admin.server_version < (4, 2, 12) or admin.server_version == (4,3,0):
	454	self.skipTest('Password initialization is broken before iRODS 4.2.12, and in 4.3.0')
	455	rodsuser = groupadmin = None
	456	rodsuser_name = 'bob'
	457	rodsuser_password = 'random_password'
	458	try:
	459	# Create a groupadmin.
	460	groupadmin, groupadmin_session = self.create_groupadmin_user_and_session('groupadmin_428')
	461
	462	# Use the groupadmin to create a new user initialized with a known password; then, test the
	463	# new user/password combination by logging in and grabbing the home collection object.
	464	with groupadmin_session:
	465	rodsuser = groupadmin_session.users.create_with_password(rodsuser_name, rodsuser_password)
	466	with iRODSSession(user = rodsuser_name,
	467	password = rodsuser_password,
	468	host = admin.host, port = admin.port, zone = admin.zone) as rodsuser_session:
	469	rodsuser_session.collections.get(helpers.home_collection(rodsuser_session))
	470	finally:
	471	if groupadmin:
	472	groupadmin.remove()
	473	# NB: Although created by a groupadmin, the rodsuser must be removed by a rodsadmin.
	474	if rodsuser:
	475	admin.users.remove(rodsuser.name)
	476
	477	def test_demonstrating_database_password_corruption_in_modify_password__issue_448(self):
	478	ses = helpers.make_session()
	479	if ses.users.get( ses.username ).type != 'rodsadmin':
	480	self.skipTest( 'Only a rodsadmin may run this test.')
	481
	482	user_alice = None
	483	extra="."
	484	NEWPASS='1234567890123456789012345'
	485	OLDPASS='123456789012345678901234'
	486
	487	try:
	488	user_alice = ses.users.create('alice', 'rodsuser')
	489	ses.users.modify('alice', 'password', OLDPASS)
	490
	491	def alice_login(pw):
	492	with iRODSSession(user='alice', password=pw, host=ses.host, port=ses.port, zone=ses.zone) as alice:
	493	home = helpers.home_collection( alice )
	494	alice.collections.get(home)
	495
	496	alice_login(OLDPASS)
	497
	498	# Alice makes an unsuccessful attempt at changing her own password.
	499	with iRODSSession(user='alice', password=OLDPASS, host=ses.host, port=ses.port, zone=ses.zone) as alice:
	500	me = alice.users.get(alice.username)
	501	try:
	502	me.modify_password(OLDPASS+extra, NEWPASS)
	503	except Bad_password_change_parameter:
	504	pass
	505
	506	# Whether the server raises an error or not, one would expect that either password in the catalog should
	507	# at this point have the desired new value, or it should have been left as it was.
	508	# Unfortunately the iRODS server does not guarantee this during the unfiltered use of USER_ADMIN_AN.
	509	# (See https://github.com/irods/python-irodsclient/issues/448 .)
	510	# Thus, here we make two trials to ensure the condition of integrity. If neither the old nor the new
	511	# password works, 'successes' will stay 0 and the assertion below will fail. This would indicate the
	512	# corruption of the database-stored password that is the subject of this test.
	513	successes = 0
	514	for password in (OLDPASS, NEWPASS):
	515	try:
	516	alice_login(password)
	517	except ex.CAT_INVALID_AUTHENTICATION:
	518	pass
	519	else:
	520	successes += 1
	521	self.assertGreater(successes, 0)
	522
	523	finally:
	524	if user_alice:
	525	user_alice.remove()
	526
	527
338	528	if __name__ == '__main__':
339	529	# let the tests find the parent irods lib
340	530	sys.path.insert(0, os.path.abspath('../..'))

+1

-1

irods/test/zone_test.py less more

41	41	except CollectionDoesNotExist:
42	42	continue
43	43	perm = iRODSAccess( 'own', p.path, session.username, session.zone)
44		session.permissions.set( perm, admin=True)
	44	session.acls.set( perm, admin=True)
45	45	p.remove(force=True)
46	46
47	47

+10

-5

irods/user.py less more

0	0	from __future__ import absolute_import
1		from irods.models import User, UserGroup, UserAuth
	1	from irods.models import User, Group, UserAuth
2	2	from irods.meta import iRODSMetaCollection
3	3	from irods.exception import NoResultFound
4	4
5	5	_Not_Defined = ()
	6
	7	class Bad_password_change_parameter(Exception): pass
6	8
7	9	class iRODSUser(object):
8	10

61	63	return self.manager.temp_password_for_user(self.name)
62	64
63	65
64		class iRODSUserGroup(object):
	66	class iRODSGroup(object):
65	67
66	68	def __init__(self, manager, result=None):
67	69	self.manager = manager
68	70	if result:
69		self.id = result[UserGroup.id]
70		self.name = result[UserGroup.name]
	71	self.id = result[Group.id]
	72	self.name = result[Group.name]
71	73	self._meta = None
72	74
73	75	def __repr__(self):
74		return "<iRODSUserGroup {id} {name}>".format(**vars(self))
	76	return "<iRODSGroup {id} {name}>".format(**vars(self))
75	77
76	78	def remove(self):
77	79	self.manager.remove(self.name)

96	98	def hasmember(self, user_name):
97	99	member_names = [user.name for user in self.members]
98	100	return user_name in member_names
	101
	102	# The iRODSUserGroup is now renamed iRODSGroup, but we'll keep the deprecated name around for now.
	103	iRODSUserGroup = iRODSGroup

+1

-1

irods/version.py less more

0		__version__ = '1.1.5'
	0	__version__ = '1.1.8'